本代码演示去电影网址上抓取电影网址和电影名称的数据,并保存到本机到excel表格中
操作步骤如下:
一、去百度上搜索查询要爬去的电影网址:
搜索到的电影网址为:
http://dianying.2345.com/list/guzhuang------.html,即我们今天要通过python进行get请求的网址
二、开始测试爬取网址:
1、在360极速浏览器输入网址:
http://dianying.2345.com/list/guzhuang------.html。
2、按F12查看网址的源代码。
3、按住F5加载网页源代码。
4、找到左下角的Network,点击↖箭头,选择对应的电影,如下图所示:
相当于已经得出了如下结论:
所有的电影网址,标题信息都在<em class='emTit'>标签里。
三、去pycharm里编写源代码
1、同时按alt+insert键,新建一个使用find_all匹配电影信息.py的文件。
2、根据以下3个步骤开始编写爬取网页源代码的代码:
①导入相应的库
②定义函数对网页进行get请求,并解析网址,并保存数据到excel中
③调用函数
代码示例如下:
在pycharm同时按住ctrl+shift+F10运行代码,代码运行结果如下:
[<em class='emTit'><a data-ajax83='ys_dy_list_title_148874' href='
//dianying.2345.com/detail/148874.html' target='_blank' title='王朝的女人杨贵妃'>王朝的女人・杨贵妃</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_35077' href='
//dianying.2345.com/detail/35077.html' target='_blank' title='龙门飞甲'>龙门飞甲</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_201821' href='
//dianying.2345.com/detail/201821.html' target='_blank' title='开封降魔记'>开封降魔记</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_59747' href='
//dianying.2345.com/detail/59747.html' target='_blank' title='黄飞鸿1壮志凌云'>黄飞鸿1:壮志凌云</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_199282' href='
//dianying.2345.com/detail/199282.html' target='_blank' title='神谕通天'>神谕通天</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_39273' href='
//dianying.2345.com/detail/39273.html' target='_blank' title='笑傲江湖3东方不败风云再起'>笑傲江湖3:东方不败…</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_202433' href='
//dianying.2345.com/detail/202433.html' target='_blank' title='封魔传'>封魔传</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_30550' href='
//dianying.2345.com/detail/30550.html' target='_blank' title='恶虎村'>恶虎村</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_190993' href='
//dianying.2345.com/detail/190993.html' target='_blank' title='少林与武当'>少林与武当</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_19317' href='
//dianying.2345.com/detail/19317.html' target='_blank' title='镖行天下前传之决战天涯'>镖行天下前传之决战…</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_198268' href='
//dianying.2345.com/detail/198268.html' target='_blank' title='大唐狐妖传'>大唐狐妖传</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_164786' href='
//dianying.2345.com/detail/164786.html' target='_blank' title='轩辕大帝'>轩辕大帝</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_141359' href='
//dianying.2345.com/detail/141359.html' target='_blank' title='白幽灵传奇之绝命逃亡'>白幽灵传奇之绝命逃…</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_21034' href='
//dianying.2345.com/detail/21034.html' target='_blank' title='火烧圆明园'>火烧圆明园</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_190410' href='
//dianying.2345.com/detail/190410.html' target='_blank' title='埋伏1973'>埋伏(1973)</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_120632' href='
//dianying.2345.com/detail/120632.html' target='_blank' title='绣春刀'>绣春刀</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_84172' href='
//dianying.2345.com/detail/84172.html' target='_blank' title='止杀令'>止杀令</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_21898' href='
//dianying.2345.com/detail/21898.html' target='_blank' title='黄飞鸿6西域雄狮'>黄飞鸿6:西域雄狮</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_200132' href='
//dianying.2345.com/detail/200132.html' target='_blank' title='龙族的反击'>龙族的反击</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_197808' href='
//dianying.2345.com/detail/197808.html' target='_blank' title='白夜侠'>白夜侠</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_323' href='
//dianying.2345.com/detail/323.html' target='_blank' title='倚天屠龙记之魔教教主'>倚天屠龙记之魔教教…</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_116829' href='
//dianying.2345.com/detail/116829.html' target='_blank' title='天将雄师'>天将雄师</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_50331' href='
//dianying.2345.com/detail/50331.html' target='_blank' title='大罗剑侠'>大罗剑侠</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_36044' href='
//dianying.2345.com/detail/36044.html' target='_blank' title='杨门女将之军令如山'>杨门女将之军令如山…</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_165827' href='
//dianying.2345.com/detail/165827.html' target='_blank' title='奇侠'>奇侠</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_202024' href='
//dianying.2345.com/detail/202024.html' target='_blank' title='等到烟暖雨收'>等到烟暖雨收</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_31845' href='
//dianying.2345.com/detail/31845.html' target='_blank' title='古墓荒斋'>古墓荒斋</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_50332' href='
//dianying.2345.com/detail/50332.html' target='_blank' title='黑灵官'>黑灵官</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_200206' href='
//dianying.2345.com/detail/200206.html' target='_blank' title='铁锅传奇'>铁锅传奇</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_65349' href='
//dianying.2345.com/detail/65349.html' target='_blank' title='花漾'>花漾</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_30492' href='
//dianying.2345.com/detail/30492.html' target='_blank' title='鹰王'>鹰王</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_198270' href='
//dianying.2345.com/detail/198270.html' target='_blank' title='皇上命我来选妃'>皇上命我来选妃</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_19263' href='
//dianying.2345.com/detail/19263.html' target='_blank' title='镖行天下前传之终极任务'>镖行天下前传之终极…</a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_165826' href='
//dianying.2345.com/detail/165826.html' target='_blank' title='绣春刀修罗战场'>绣春刀・修罗战场 </a></em>, <em class='emTit'><a data-ajax83='ys_dy_list_title_196850' href='
//dianying.2345.com/detail/196850.html' target='_blank' title='维京王者之战'>维京:王者之战</a></em>]
<em class='emTit'><a data-ajax83='ys_dy_list_title_148874' href='
//dianying.2345.com/detail/148874.html' target='_blank' title='王朝的女人杨贵妃'>王朝的女人・杨贵妃</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_35077' href='
//dianying.2345.com/detail/35077.html' target='_blank' title='龙门飞甲'>龙门飞甲</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_201821' href='
//dianying.2345.com/detail/201821.html' target='_blank' title='开封降魔记'>开封降魔记</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_59747' href='
//dianying.2345.com/detail/59747.html' target='_blank' title='黄飞鸿1壮志凌云'>黄飞鸿1:壮志凌云</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_199282' href='
//dianying.2345.com/detail/199282.html' target='_blank' title='神谕通天'>神谕通天</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_39273' href='
//dianying.2345.com/detail/39273.html' target='_blank' title='笑傲江湖3东方不败风云再起'>笑傲江湖3:东方不败…</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_202433' href='
//dianying.2345.com/detail/202433.html' target='_blank' title='封魔传'>封魔传</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_30550' href='
//dianying.2345.com/detail/30550.html' target='_blank' title='恶虎村'>恶虎村</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_190993' href='
//dianying.2345.com/detail/190993.html' target='_blank' title='少林与武当'>少林与武当</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_19317' href='
//dianying.2345.com/detail/19317.html' target='_blank' title='镖行天下前传之决战天涯'>镖行天下前传之决战…</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_198268' href='
//dianying.2345.com/detail/198268.html' target='_blank' title='大唐狐妖传'>大唐狐妖传</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_164786' href='
//dianying.2345.com/detail/164786.html' target='_blank' title='轩辕大帝'>轩辕大帝</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_141359' href='
//dianying.2345.com/detail/141359.html' target='_blank' title='白幽灵传奇之绝命逃亡'>白幽灵传奇之绝命逃…</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_21034' href='
//dianying.2345.com/detail/21034.html' target='_blank' title='火烧圆明园'>火烧圆明园</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_190410' href='
//dianying.2345.com/detail/190410.html' target='_blank' title='埋伏1973'>埋伏(1973)</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_120632' href='
//dianying.2345.com/detail/120632.html' target='_blank' title='绣春刀'>绣春刀</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_84172' href='
//dianying.2345.com/detail/84172.html' target='_blank' title='止杀令'>止杀令</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_21898' href='
//dianying.2345.com/detail/21898.html' target='_blank' title='黄飞鸿6西域雄狮'>黄飞鸿6:西域雄狮</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_200132' href='
//dianying.2345.com/detail/200132.html' target='_blank' title='龙族的反击'>龙族的反击</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_197808' href='
//dianying.2345.com/detail/197808.html' target='_blank' title='白夜侠'>白夜侠</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_323' href='
//dianying.2345.com/detail/323.html' target='_blank' title='倚天屠龙记之魔教教主'>倚天屠龙记之魔教教…</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_116829' href='
//dianying.2345.com/detail/116829.html' target='_blank' title='天将雄师'>天将雄师</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_50331' href='
//dianying.2345.com/detail/50331.html' target='_blank' title='大罗剑侠'>大罗剑侠</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_36044' href='
//dianying.2345.com/detail/36044.html' target='_blank' title='杨门女将之军令如山'>杨门女将之军令如山…</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_165827' href='
//dianying.2345.com/detail/165827.html' target='_blank' title='奇侠'>奇侠</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_202024' href='
//dianying.2345.com/detail/202024.html' target='_blank' title='等到烟暖雨收'>等到烟暖雨收</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_31845' href='
//dianying.2345.com/detail/31845.html' target='_blank' title='古墓荒斋'>古墓荒斋</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_50332' href='
//dianying.2345.com/detail/50332.html' target='_blank' title='黑灵官'>黑灵官</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_200206' href='
//dianying.2345.com/detail/200206.html' target='_blank' title='铁锅传奇'>铁锅传奇</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_65349' href='
//dianying.2345.com/detail/65349.html' target='_blank' title='花漾'>花漾</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_30492' href='
//dianying.2345.com/detail/30492.html' target='_blank' title='鹰王'>鹰王</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_198270' href='
//dianying.2345.com/detail/198270.html' target='_blank' title='皇上命我来选妃'>皇上命我来选妃</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_19263' href='
//dianying.2345.com/detail/19263.html' target='_blank' title='镖行天下前传之终极任务'>镖行天下前传之终极…</a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_165826' href='
//dianying.2345.com/detail/165826.html' target='_blank' title='绣春刀修罗战场'>绣春刀・修罗战场 </a></em>
<em class='emTit'><a data-ajax83='ys_dy_list_title_196850' href='
//dianying.2345.com/detail/196850.html' target='_blank' title='维京王者之战'>维京:王者之战</a></em>
list1: ['http://dianying.2345.com/detail/148874.html', 'http://dianying.2345.com/detail/35077.html', 'http://dianying.2345.com/detail/201821.html', 'http://dianying.2345.com/detail/59747.html', 'http://dianying.2345.com/detail/199282.html', 'http://dianying.2345.com/detail/39273.html', 'http://dianying.2345.com/detail/202433.html', 'http://dianying.2345.com/detail/30550.html', 'http://dianying.2345.com/detail/190993.html', 'http://dianying.2345.com/detail/19317.html', 'http://dianying.2345.com/detail/198268.html', 'http://dianying.2345.com/detail/164786.html', 'http://dianying.2345.com/detail/141359.html', 'http://dianying.2345.com/detail/21034.html', 'http://dianying.2345.com/detail/190410.html', 'http://dianying.2345.com/detail/120632.html', 'http://dianying.2345.com/detail/84172.html', 'http://dianying.2345.com/detail/21898.html', 'http://dianying.2345.com/detail/200132.html', 'http://dianying.2345.com/detail/197808.html', 'http://dianying.2345.com/detail/323.html', 'http://dianying.2345.com/detail/116829.html', 'http://dianying.2345.com/detail/50331.html', 'http://dianying.2345.com/detail/36044.html', 'http://dianying.2345.com/detail/165827.html', 'http://dianying.2345.com/detail/202024.html', 'http://dianying.2345.com/detail/31845.html', 'http://dianying.2345.com/detail/50332.html', 'http://dianying.2345.com/detail/200206.html', 'http://dianying.2345.com/detail/65349.html', 'http://dianying.2345.com/detail/30492.html', 'http://dianying.2345.com/detail/198270.html', 'http://dianying.2345.com/detail/19263.html', 'http://dianying.2345.com/detail/165826.html', 'http://dianying.2345.com/detail/196850.html']
list2: ['王朝的女人・杨贵妃', '龙门飞甲', '开封降魔记', '黄飞鸿1:壮志凌云', '神谕通天', '笑傲江湖3:东方不败…', '封魔传', '恶虎村', '少林与武当', '镖行天下前传之决战…', '大唐狐妖传', '轩辕大帝', '白幽灵传奇之绝命逃…', '火烧圆明园', '埋伏(1973)', '绣春刀', '止杀令', '黄飞鸿6:西域雄狮', '龙族的反击', '白夜侠', '倚天屠龙记之魔教教…', '天将雄师', '大罗剑侠', '杨门女将之军令如山…', '奇侠', '等到烟暖雨收', '古墓荒斋', '黑灵官', '铁锅传奇', '花漾', '鹰王', '皇上命我来选妃', '镖行天下前传之终极…', '绣春刀・修罗战场 ', '维京:王者之战']
pycharm代码图片示例如下:
pycharm代码图
检查网址的电影信息图片
检查网址的电影信息图片
保存到本机的EXCEL表格图片
保存到本机的EXCEL表格图片
联系客服