|
使用requests+代理获取某个网站内容时，一直乱码，源代码是这样的：
headers:
"headers": {
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"accept-encoding": "gzip, deflate, br",
"accept-language": "zh-CN,zh;q=0.9",
"cache-control": "max-age=0",
"sec-ch-ua": "\" Not;A Brand\";v=\"99\", \"Google Chrome\";v=\"97\", \"Chromium\";v=\"97\"",
"sec-ch-ua-mobile": "?0",
"sec-ch-ua-platform": "\"Windows\"",
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "same-origin",
"sec-fetch-user": "?1",
"upgrade-insecure-requests": "1",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36"
}
调用的时候是这样的：
proxies = {
"HTTP": "http://{}:{}@{}:{}".format(pconfig['proxyUser'], pconfig['proxyPass'], pconfig['proxyHost'], pconfig['proxyPort']),
"HTTPS": "https://{}:{}@{}:{}".format(pconfig['proxyUser'], pconfig['proxyPass'], pconfig['proxyHost'], pconfig['proxyPort'])
}
result = requests.get(url=url, headers=headers, timeout=timeout, proxies=proxies, verify=False)
if result is not None and result.status_code == 200:
errMsg = ""
urlhtml = result.text
realurl = result.url # 重定向地址
break
else:
但是获取到的结果是这样的：
result.content= "b'c\x00\xa0a\x00@6s\xd6\x1b\x1f\x7f\xd5\x07\x14\x17\x17C\x07\x88\xaa\xa3&\xea\x18r\xb6\xb2|\xff\x97\tr\xc0G\xd4\xbdT=\xd0\x13\xd9\xdd\xf9WRz\xa1V\xe6\xcf\xfeMoX\x9b]\x82y\xb1}\x9e\x8a!h\x85\x0e8D\x01\x8fH\x1e\xcbu\x91?\x97\x84\x08\xe2%n?D9z\xf4\xf6\xe1\xa7\xef\xef\x1e\xc3\xb3O\xaf_\xd5P....."
result.text="c �a @6s��C���&�r��|��?? ?r�G?T=����WRz�V���MoX�]�y�}��!h�8D�H�u�?���%n?D9z�����óO�_�P����+�������?�����?? ?�h����0�dV!�>�aH�����q8x�������36ZD��cC"M��aSC�H�7@k"��=��b�~N(�s/�Fj�!G3�$а�w���O���n��GQщ?......"
看了一下result.apparent_encoding，然而：result.apparent_encoding=None? ?(吐血！)
当时搜了各种解决方案(以下给大家参考，也许可用)：
你的apparent_encoding 还准确么？ - 灰信网（软件开发博客聚合）
Python requests响应内容乱码解决方案 | 漏洞人生
解决Requests中文乱码_chaowanghn的博客-CSDN博客_requests 乱码
https://segmentfault.com/a/1190000012984536
但是上述办法对我毫无效果，直到第二天看到这篇文章：
requests 中文编码错误_chaoge_dgqb的博客-CSDN博客_requests编码错误
终于发现我的headers设置了一个多余的：
"accept-encoding": "gzip, deflate, br",
把它删掉之后，就立马好了。
所以有时候少即是多....(说点假道理，其实多用不同方式搜索问题，才是正确解法)
|