test di selenio Python rimane incastrato in urlopen

Question

mia app si basa su:

• Python 3
• Django 1.8
• Weasyprint
• Selenio

Funziona senza problemi su ambiente dev e la produzione , ma non durante il test con il selenio.

Utilizzando weasyprint, creo un PDF da HTML, questa libreria utilizza urllib per scaricare CSS (ad esempio http://localhost:8081/static/lib/bootstrap/css/bootstrap.min.css), ma si blocca (nessun errore, solo bloccato) durante l'apertura di questi.

Se inserisco questo URL direttamente nel browser mentre è impiccato, viene visualizzato il CSS.

comando utilizzato:

./manage.py test tests.test_account.HomeNewVisitorTest 

parte rilevante del test:

from selenium import webdriver 

class HomeNewVisitorTest(StaticLiveServerTestCase): 
    def setUp(self): 
     if TEST_ENV_FIREFOX: 
      self.driver = webdriver.Firefox() 
     else: 
      self.driver = webdriver.PhantomJS() 
     self.driver.set_window_size(1440, 900) 

    def tearDown(self): 
     try: 
      path = 'worksites/' + self.worksite_name.lower() 
      os.rmdir(settings.MEDIA_ROOT + path) 
     except FileNotFoundError: 
      pass 
     super().tearDown() 

    def test(self): 
     d = self.driver 
     d.get(self.get_full_url('home')) 
     d.find_element_by_css_selector('.btn-success[type=submit]').click() 

A mio avviso:

# Generate PDF for contact directory 
    template = get_template("pdf/annuaire.html") 
    context = {"worksite": worksite} 
    html = template.render(RequestContext(self.request, context)) 
    base_url = self.request.build_absolute_uri("/") 
    pdf = weasyprint.HTML(string=html, base_url=base_url) 
    pdf.write_pdf(directory + '/annuaire.pdf') 

Ecco la discarica filo quando bloccato:

Fatal Python error: Aborted 

Thread 0x0000000106f92000 (most recent call first): 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socket.py", line 374 in readinto 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 313 in _read_status 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 351 in begin 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1171 in getresponse 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1185 in do_open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1210 in http_open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 441 in _call_chain 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 481 in _open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 463 in open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 161 in urlopen 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/urls.py", line 276 in default_url_fetcher 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/urls.py", line 311 in fetch 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/contextlib.py", line 59 in __enter__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/__init__.py", line 297 in _select_source 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/contextlib.py", line 59 in __enter__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/__init__.py", line 223 in __init__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/css/__init__.py", line 198 in find_stylesheets 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/css/__init__.py", line 448 in get_all_computed_styles 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/document.py", line 312 in _render 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/__init__.py", line 132 in render 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/weasyprint/__init__.py", line 164 in write_pdf 
    File "/Users/sebcorbin/Sites/planexo/worksite/views.py", line 111 in done 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/formtools/wizard/views.py", line 357 in render_done 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/formtools/wizard/views.py", line 730 in render_done 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/formtools/wizard/views.py", line 300 in post 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/formtools/wizard/views.py", line 686 in post 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/views/generic/base.py", line 89 in dispatch 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/formtools/wizard/views.py", line 237 in dispatch 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/views/generic/base.py", line 71 in view 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/contrib/auth/decorators.py", line 22 in _wrapped_view 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/handlers/base.py", line 132 in get_response 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/handlers/wsgi.py", line 189 in __call__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/test/testcases.py", line 1099 in __call__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/contrib/staticfiles/handlers.py", line 63 in __call__ 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/wsgiref/handlers.py", line 137 in run 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/servers/basehttp.py", line 182 in handle 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 673 in __init__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/servers/basehttp.py", line 102 in __init__ 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 344 in finish_request 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 331 in process_request 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 305 in _handle_request_noblock 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socketserver.py", line 238 in serve_forever 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/test/testcases.py", line 1182 in run 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/threading.py", line 920 in _bootstrap_inner 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/threading.py", line 888 in _bootstrap 

Current thread 0x00007fff7996a300 (most recent call first): 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/socket.py", line 374 in readinto 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 313 in _read_status 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 351 in begin 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/http/client.py", line 1171 in getresponse 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1185 in do_open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 1210 in http_open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 441 in _call_chain 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 481 in _open 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/urllib/request.py", line 463 in open 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/selenium/webdriver/remote/remote_connection.py", line 457 in _request 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/selenium/webdriver/remote/remote_connection.py", line 389 in execute 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/selenium/webdriver/remote/webdriver.py", line 191 in execute 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/selenium/webdriver/remote/webelement.py", line 447 in _execute 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/selenium/webdriver/remote/webelement.py", line 68 in click 
    File "/Users/sebcorbin/Sites/planexo/tests/test_account.py", line 203 in _test_worksite_form 
    File "/Users/sebcorbin/Sites/planexo/tests/test_account.py", line 36 in test 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/case.py", line 577 in run 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/case.py", line 625 in __call__ 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/test/testcases.py", line 186 in __call__ 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/suite.py", line 122 in run 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/suite.py", line 84 in __call__ 
    File "/usr/local/Cellar/python3/3.4.3/Frameworks/Python.framework/Versions/3.4/lib/python3.4/unittest/runner.py", line 168 in run 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/test/runner.py", line 178 in run_suite 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/test/runner.py", line 211 in run_tests 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/commands/test.py", line 90 in handle 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/base.py", line 441 in execute 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/commands/test.py", line 74 in execute 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/base.py", line 390 in run_from_argv 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/commands/test.py", line 30 in run_from_argv 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/__init__.py", line 330 in execute 
    File "/Users/sebcorbin/.virtualenvs/planexo-py3/lib/python3.4/site-packages/django/core/management/__init__.py", line 338 in execute_from_command_line 
    File "./manage.py", line 10 in <module> 

Answer 1

Grazie a tutti voi per individuare il problema, ho preso ispirazione questo articolo http://nedbatchelder.com/blog/201103/quick_and_dirty_multithreaded_django_dev_server.html e modificati mia manage.py di file:

#!/usr/bin/env python 
from http import server 
import os 
from socketserver import ThreadingMixIn 
import sys 


def monkey_patch_test_server(): 
    # This monkey-patches HTTPServer to create a base HTTPServer class that 
    # supports multithreading 
    originalhttpserver = server.HTTPServer 

    class ThreadedHTTPServer(ThreadingMixIn, originalhttpserver): 
     def __init__(self, server_address, RequestHandlerClass=None): 
      originalhttpserver.__init__(self, server_address, 
             RequestHandlerClass) 

    server.HTTPServer = ThreadedHTTPServer 


if __name__ == "__main__": 
    os.environ.setdefault("DJANGO_SETTINGS_MODULE", "planexo.settings") 

    from django.core.management import execute_from_command_line 

    if sys.argv[1] == 'test': 
     monkey_patch_test_server() 

    execute_from_command_line(sys.argv) 

Ora funziona!

Answer 2

Utilizzare il modulo Python faulthandler per stampare il traceback di tutti i thread appesi. Questo ti indicherà le funzioni effettive in cui sta avvenendo l'impiccagione.

Se necessario, modifica la domanda e aggiungi le informazioni pertinenti.

Altrimenti suppongo che tu stia utilizzando il server di test/sviluppo Django in modo tale che possa gestire solo una richiesta alla volta. La richiesta HTTP iniziale al server dev avvia una richiesta per richiedere dati da sé, che il server dev Django non è in grado di eseguire. Ma questa è solo una supposizione.

Manca anche il codice sorgente pertinente dalla domanda.

Answer 3

Utilizzare il modulo faulthandler. Forse può anche essere che Django, può solo prendere 1 richiesta.

Answer 4

io condividere il mio caso:

ho avuto questo problema con i miei test selenio Django Django (== 1.7.12 e selenio == 2.53.1) con ChromeDriver 2.21.371459 e Google Chrome 48.0.2564.116 .

Sono stato in grado di isolare il problema. Nel mio caso stava accadendo solo per le pagine che fanno riferimento a un file statico (un'immagine in un tag HTML <img> per esempio http://cdn.local.myproject.net/static/myimage.png) sul mio dominio cdn locale personalizzato. Il problema non era presente se avessi usato un percorso relativo "/ static/myimage.png" o localhost "http://127.0.0.1/static/myimage.png" così ho pensato che fosse un problema di DNS

sono stato in grado di aggirare il problema utilizzando l'opzione --dns-prefetch-disable di cromo

Esempio in Python:..

from selenium.webdriver import Chrome 
from selenium.webdriver.chrome.options import Options 

chrome_options = Options() 
chrome_options.add_argument('--dns-prefetch-disable') 
driver = Chrome(chrome_options=options) 

Non so se questo è il caso generale, ma spero che possa aiutare qualcuno di voi