Python中设置Selenium + Chrome代理IP（密码验证）爬数据作者：数据无忧 时间：2020-09-18 13:24:53 |
原文stackoverflow地址：https://stackoverflow.com/questions/29983106/how-can-i-set-proxy-with-authentication-in-selenium-chrome-web-driver-using-pyth#answer-30953780 Selenium+Chrome认证代理不能通过options处理。只能换个方法使用扩展解决，下面是Python代码 # -*- coding: utf-8 -*- # @Time : 2018/10/05 9:50 # @Author : www.data5u.com # @Site : # @File : pubilc.py # @Software: PyCharm import string import zipfile def create_proxyauth_extension(proxy_host, proxy_port, proxy_username, proxy_password, scheme='http', plugin_path=None): """代理认证插件 args: proxy_host (str): 你的代理地址或者域名（str类型） proxy_port (int): 代理端口号（int类型） proxy_username (str):用户名（字符串） proxy_password (str): 密码 （字符串） kwargs: scheme (str): 代理方式 默认http plugin_path (str): 扩展的绝对路径 return str -> plugin_path """ if plugin_path is None: plugin_path = 'vimm_chrome_proxyauth_plugin.zip' manifest_json = """ { "version": "1.0.0", "manifest_version": 2, "name": "Chrome Proxy", "permissions": [ "proxy", "tabs", "unlimitedStorage", "storage", "<all_urls>", "webRequest", "webRequestBlocking" ], "background": { "s": ["background.js"] }, "minimum_chrome_version":"22.0.0" } """ background_js = string.Template( """ var config = { mode: "fixed_servers", rules: { singleProxy: { scheme: "${scheme}", host: "${host}", port: parseInt(${port}) }, bypassList: ["foobar.com"] } }; chrome.proxy.settings.set({value: config, scope: "regular"}, function() {}); function callbackFn(details) { return { authCredentials: { username: "${username}", password: "${password}" } }; } chrome.webRequest.onAuthRequired.addListener( callbackFn, {urls: ["<all_urls>"]}, ['blocking'] ); """ ).substitute( host=proxy_host, port=proxy_port, username=proxy_username, password=proxy_password, scheme=scheme, ) with zipfile.ZipFile(plugin_path, 'w') as zp: zp.writestr("manifest.json", manifest_json) zp.writestr("background.js", background_js) return plugin_path 使用方法如下： from selenium import webdriver from common.pubilc import create_proxyauth_extension proxyauth_plugin_path = create_proxyauth_extension( proxy_host="XXXXX.com", proxy_port=9020, proxy_username="XXXXXXX", proxy_password="XXXXXXX" ) co = webdriver.ChromeOptions() # co.add_argument("--start-maximized") co.add_extension(proxyauth_plugin_path) driver = webdriver.Chrome(executable_path="C:\chromedriver.exe", chrome_options=co) driver.get("http://ip138.com/") print(driver.page_source) 如果代理IP不需要密码验证，那直接用下面的代码就行了： options = webdriver.ChromeOptions() options.add_argument('--proxy-server=http://ip:port') driver = webdriver.Chrome(executable_path="C:\chromedriver.exe", chrome_options=0ptions) driver.get("http://ip138.com/") print(driver.page_source)
福利：现在通过开发者购买代理IP，购买后联系客服最高可获
返现20%
到你的支付宝（最低35元，最高1440元）
无忧代理IP(www.data5u.com)原创文章，转载请注明出处。
|
