python - 数据收集时的IP阻止

Question

I have the code here. I want to help that how can I set proxy for this?
I have Api of proxy server, I just want to set up that on each call it call my api.

I have added paused but didn't worked.


<h1> This is source folder containing exccel file</h1>
source = "Data"
dir_list = os.listdir(source)





def geturl(searchtext):
    query = searchtext
    for url in search(query, tld="co.in", num=1, stop=1, pause=1):
        return url

    <h1> This is source folder containing exccel file</h1>

def writeurl(value, description, url, file,Col):

    file.write( Col, 0, value )
    file.write( Col, 1, description )
    file.write( Col, 2, url )

for i in range(len(dir_list)):
    filename = dir_list[i]
    wbr = Workbook()
    sheet1 = wbr.add_sheet( 'Sheet 1' )
    wb = xlrd.open_workbook('Data/' + filename)
    sheet = wb.sheet_by_index(0)
    i=0
    count=0
    for x in range(sheet.nrows):
       # if i<=10:
        if x == 0:

            writeurl( sheet.cell_value( x, 0 ), sheet.cell_value( x, 1 ), sheet.cell_value( x, 2 ), sheet1, x )
            <h1> This is source folder containing exccel file</h1>

＃CompanyList.append（UrlSearch.CompanyDescription（sheet.cell_value（x，0），sheet.cell_value（x，1），sheet.cell_value（x，2）））
            其他：
                   尝试：
                    网址= geturl（sheet.cell_value（x，1））

                #time.sleep(randint(10, 120))
                writeurl(sheet.cell_value(x, 0),sheet.cell_value(x, 1),Url,sheet1,x)
                count=count+1
                wbr.save('OutPut/' + filename.split('.')[0] + '.xls')

                if count==45:
                    count=0
                    time.sleep(1200)


                # CompanyList.append(UrlSearch.CompanyDescription(sheet.cell_value(x, 0), sheet.cell_value(x, 1), Url))
                # i = i+1
                 <h1> This is source folder containing exccel file</h1>

除了ValueError：
                      打印（“ HTTP错误429（请求太多）”）
            打印（x）

   # wbr.save('OutPut/' + filename.split('.')[0] + '.xls')

Answer 1

您可以尝试将User-Agent更改为this网站的输出。这使服务器认为您是浏览器，并且可能会使它更宽松。另外，请尝试将请求间隔得更多：可能需要5-10秒。