爬虫-网站地址及反爬机制
1.安居客:https://cd.zu.anjuke.com/fangyuan/wuhou/x1/ - requests
2.贝壳找房:https://cd.zu.ke.com/zufang/#contentList - requests
3.腾讯视频:https://v.qq.com/ - 字体反爬
4.酷狗音乐:https://www.kugou.com/ - requests
5.起点小说:https://www.qidian.com/ - 待定(看不懂)
6.抖音视频:https://www.douyin.com/ - 看不懂
7.小红书:https://www.xiaohongshu.com/ - 好像是requests
8.汽车之家:https://www.autohome.com.cn/beijing/ - requests
9.哈罗摩托:https://www.jddmoto.com/ - requests
10.驾考宝典:https://www.jiakaobaodian.com/ - requests
11.高夫:https://www.jahwa.com.cn/gf - 报错,不知原因
12.碧桂园官网:https://www.bgy.com.cn/ - requests
13.中国移动:http://www.10086.cn/index/sc/index_280_280.html - 字体反爬
14.一加手机官网:https://www.oneplus.com/cn - 字体反爬
15.中国农业银行:https://www.abchina.com/cn/ - 字体反爬
- 天涯社区:http://focus.tianya.cn/ - 字体反爬
- 华为官网:https://www.vmall.com/index_new.html - 字体反爬
- 4399游戏:http://www.4399.com/ - 字体反爬
- 前程无忧:https://www.51job.com/ - 字体反爬
- BOSS直聘:https://www.zhipin.com/chengdu/ - user-agent
|