資源簡介
利用python的scrapy爬取慕課網全站課程數據存入Mysql,json,csv

代碼片段和文件信息
#?-*-?coding:?utf-8?-*-
#引入文件
import?scrapy
class?CourseItem(scrapy.Item):
????#課程標題
????title?=?scrapy.Field()
????#課程url
????url?=?scrapy.Field()
????#課程標題圖片
????image_url?=?scrapy.Field()
????#課程描述
????introduction?=?scrapy.Field()
????#學習人數
????student?=?scrapy.Field()
????#課程標簽
????catycray?=?scrapy.Field()
????#課程難度
????degree?=?scrapy.Field()
????#課程時長
????hour?=scrapy.Field()
????#課程評分
????score=scrapy.Field()
?屬性????????????大小?????日期????時間???名稱
-----------?---------??----------?-----??----
?????目錄???????????0??2017-08-11?09:52??scrapytest\
?????文件????????8196??2017-08-11?09:52??scrapytest\.DS_Store
?????目錄???????????0??2017-08-11?09:53??__MACOSX\
?????目錄???????????0??2017-08-11?09:53??__MACOSX\scrapytest\
?????文件?????????120??2017-08-11?09:52??__MACOSX\scrapytest\._.DS_Store
?????目錄???????????0??2017-08-11?09:52??scrapytest\pic\
?????文件????????8211??2017-08-09?11:36??scrapytest\pic\??MongoDB澶嶅埗闆嗏€斿鍒墮泦鐩戞帶.jpg
?????文件????????5202??2017-08-09?11:36??scrapytest\pic\?Linux杈句漢鍏繪垚璁″垝?I.jpg
?????文件???????10290??2017-08-09?11:36??scrapytest\pic\?MongoDB澶嶅埗闆嗏€斿鐏炬牳蹇冮€変婦.jpg
?????文件????????6148??2017-08-11?09:52??scrapytest\pic\.DS_Store
?????目錄???????????0??2017-08-11?09:53??__MACOSX\scrapytest\pic\
?????文件?????????120??2017-08-11?09:52??__MACOSX\scrapytest\pic\._.DS_Store
?????文件?????????264??2017-07-19?16:49??scrapytest\scrapy.cfg
?????目錄???????????0??2017-08-10?17:27??scrapytest\scrapytest\
?????文件????????8196??2017-08-10?15:53??scrapytest\scrapytest\.DS_Store
?????目錄???????????0??2017-08-11?09:53??__MACOSX\scrapytest\scrapytest\
?????文件?????????120??2017-08-10?15:53??__MACOSX\scrapytest\scrapytest\._.DS_Store
?????文件???????????0??2017-07-11?11:02??scrapytest\scrapytest\__init__.py
?????文件?????????144??2017-07-26?11:27??scrapytest\scrapytest\__init__.pyc
?????文件?????????504??2017-08-08?17:41??scrapytest\scrapytest\CourseItems.py
?????文件?????????210??2017-08-08?17:41??__MACOSX\scrapytest\scrapytest\._CourseItems.py
?????文件?????????638??2017-08-08?18:13??scrapytest\scrapytest\CourseItems.pyc
?????文件?????????289??2017-07-19?16:49??scrapytest\scrapytest\items.py
?????文件????????1908??2017-07-19?16:49??scrapytest\scrapytest\middlewares.py
?????文件????????1690??2017-08-10?17:06??scrapytest\scrapytest\MyPipelines.py
?????文件?????????210??2017-08-10?17:06??__MACOSX\scrapytest\scrapytest\._MyPipelines.py
?????文件????????2249??2017-08-10?17:27??scrapytest\scrapytest\MyPipelines.pyc
?????文件?????????290??2017-07-19?16:49??scrapytest\scrapytest\pipelines.py
?????文件????????3623??2017-08-10?17:08??scrapytest\scrapytest\settings.py
?????文件?????????503??2017-08-10?17:27??scrapytest\scrapytest\settings.pyc
?????目錄???????????0??2017-08-10?17:27??scrapytest\scrapytest\spiders\
............此處省略11個文件信息
- 上一篇:基于python scrapy 的雙色球爬蟲
- 下一篇:python爬取視頻源代碼
評論
共有 條評論