学习scrapy 爬虫一定要知道的三种运行方式-Python 技术分享 Java技术分享 Python 爬虫技术_微信公众号:zeropython—昊天博客
# -*- coding: utf-8 -*-
"""
@Time: 2018/1/14
@Author: songhao
@微信公众号: zeropython
@File: c3.py
"""

"""
方式一:
进入项目目录,运行 scrapy3 crawl spiername
"""
# scrapy3 startproject demo demo1
# cd demo1
# scrapy3 crawl spiername


"""
方式二:这种方式可以在Pycharm中运行debug
如何单独运行 一个scrapy
"""
import sys
import os
from scrapy.cmdline import execute

# 当前文件的父目录
file_path = os.path.dirname(__file__)
print(file_path) # /Users/songhao/py/re_c

"""
sys.path是python的搜索模块的路径集,是一个list
可以在python 环境下使用sys.path.append(path)添加相关的路径,但在退出python环境后自己添加的路径就会自动消失了!
"""
sys.path.append(os.path.dirname(__file__))

# 获取文件的绝对路径
print(__file__) # /Users/songhao/py/re_c/c3.py

print(sys.path)
# ['/Users/songhao/py/re_c', '/Users/songhao/py', '/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python36.zip', '/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6', '/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload', '/Users/songhao/Library/Python/3.6/lib/python/site-packages', '/usr/local/lib/python3.6/site-packages', '/Applications/PyCharm.app/Contents/helpers/pycharm_matplotlib_backend', '/Users/songhao/py/re_c']

execute("scrapy crawl spidername".split(' '))


"""
方式三: 单独运行一个文件
scrapy3 genspider d_image macappstore.net
"""
# scrapy3 genspider d_image macappstore.net
# scrapy3 runspider d_image.py
HTTPX 基础教程-新乡seo|网站优化,网站建设_微信公众号:zeropython—昊天博客