2016-07-18 145 views
7

Sto provando a utilizzare il mio s3ninja locale con s3cmd. Ogni comando come: s3cmd ls s3://test genera le stesse eccezioni.S3Cmd non funziona con S3 Ninja

s3cfg sembra essere ok e gli endpoint chiamati sono corretti.

Qualcuno poteva usare s3ninja con s3cmd?

PS: So che l'S3 non è costoso e ci sono molti modi migliori per testare S3. Ho bisogno di S3 Ninja per altri motivi.

Ecco le informazioni di debug:

DEBUG: ConfigParser: Reading file '/Users/daniel/.s3cfg' 
DEBUG: ConfigParser: access_key->AK...17_chars...E 
DEBUG: ConfigParser: access_token-> 
DEBUG: ConfigParser: add_encoding_exts-> 
DEBUG: ConfigParser: add_headers-> 
DEBUG: ConfigParser: bucket_location->US 
DEBUG: ConfigParser: ca_certs_file-> 
DEBUG: ConfigParser: cache_file-> 
DEBUG: ConfigParser: check_ssl_certificate->True 
DEBUG: ConfigParser: check_ssl_hostname->True 
DEBUG: ConfigParser: cloudfront_host->cloudfront.amazonaws.com 
DEBUG: ConfigParser: default_mime_type->binary/octet-stream 
DEBUG: ConfigParser: delay_updates->False 
DEBUG: ConfigParser: delete_after->False 
DEBUG: ConfigParser: delete_after_fetch->False 
DEBUG: ConfigParser: delete_removed->False 
DEBUG: ConfigParser: dry_run->False 
DEBUG: ConfigParser: enable_multipart->True 
DEBUG: ConfigParser: encoding->UTF-8 
DEBUG: ConfigParser: encrypt->False 
DEBUG: ConfigParser: expiry_date-> 
DEBUG: ConfigParser: expiry_days-> 
DEBUG: ConfigParser: expiry_prefix-> 
DEBUG: ConfigParser: follow_symlinks->False 
DEBUG: ConfigParser: force->False 
DEBUG: ConfigParser: get_continue->False 
DEBUG: ConfigParser: gpg_command->None 
DEBUG: ConfigParser: gpg_decrypt->%(gpg_command)s -d --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s 
DEBUG: ConfigParser: gpg_encrypt->%(gpg_command)s -c --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s 
DEBUG: ConfigParser: gpg_passphrase->...-3_chars... 
DEBUG: ConfigParser: guess_mime_type->True 
DEBUG: ConfigParser: host_base->s3.mylocalaws.com:9444/s3 
DEBUG: ConfigParser: host_bucket->s3.mylocalaws.com:9444/s3/%(bucket)s 
DEBUG: ConfigParser: human_readable_sizes->False 
DEBUG: ConfigParser: invalidate_default_index_on_cf->False 
DEBUG: ConfigParser: invalidate_default_index_root_on_cf->True 
DEBUG: ConfigParser: invalidate_on_cf->False 
DEBUG: ConfigParser: kms_key-> 
DEBUG: ConfigParser: limitrate->0 
DEBUG: ConfigParser: list_md5->False 
DEBUG: ConfigParser: log_target_prefix-> 
DEBUG: ConfigParser: long_listing->False 
DEBUG: ConfigParser: max_delete->-1 
DEBUG: ConfigParser: mime_type-> 
DEBUG: ConfigParser: multipart_chunk_size_mb->15 
DEBUG: ConfigParser: preserve_attrs->True 
DEBUG: ConfigParser: progress_meter->True 
DEBUG: ConfigParser: proxy_host-> 
DEBUG: ConfigParser: proxy_port->0 
DEBUG: ConfigParser: put_continue->False 
DEBUG: ConfigParser: recursive->False 
DEBUG: ConfigParser: recv_chunk->65536 
DEBUG: ConfigParser: reduced_redundancy->False 
DEBUG: ConfigParser: requester_pays->False 
DEBUG: ConfigParser: restore_days->1 
DEBUG: ConfigParser: secret_key->wJ...37_chars...Y 
DEBUG: ConfigParser: send_chunk->65536 
DEBUG: ConfigParser: server_side_encryption->False 
DEBUG: ConfigParser: signature_v2->False 
DEBUG: ConfigParser: simpledb_host->sdb.amazonaws.com 
DEBUG: ConfigParser: skip_existing->False 
DEBUG: ConfigParser: socket_timeout->300 
DEBUG: ConfigParser: stop_on_error->False 
DEBUG: ConfigParser: storage_class-> 
DEBUG: ConfigParser: urlencoding_mode->normal 
DEBUG: ConfigParser: use_https->False 
DEBUG: ConfigParser: use_mime_magic->True 
DEBUG: ConfigParser: verbosity->WARNING 
DEBUG: ConfigParser: website_endpoint->http://%(bucket)s.s3-website-%(location)s.amazonaws.com/ 
DEBUG: ConfigParser: website_error-> 
DEBUG: ConfigParser: website_index->index.html 
DEBUG: Updating Config.Config cache_file -> 
DEBUG: Updating Config.Config follow_symlinks -> False 
DEBUG: Updating Config.Config verbosity -> 10 
DEBUG: Unicodising 'ls' using UTF-8 
DEBUG: Unicodising 's3://test' using UTF-8 
DEBUG: Command: ls 
DEBUG: Bucket 's3://test': 
DEBUG: CreateRequest: resource[uri]=/ 
DEBUG: Using signature v4 
DEBUG: get_hostname(test): s3.mylocalaws.com:9444/s3/test 
DEBUG: canonical_headers = host:s3.mylocalaws.com:9444/s3/test 
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 
x-amz-date:20160718T104632Z 

DEBUG: Canonical Request: 
GET 
/
delimiter=%2F 
host:s3.mylocalaws.com:9444/s3/test 
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 
x-amz-date:20160718T104632Z 

host;x-amz-content-sha256;x-amz-date 
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 
---------------------- 
DEBUG: signature-v4 headers: {'x-amz-content-sha256': 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 'Authorization': 'AWS4-HMAC-SHA256 Credential=AKIAIOSFODNN7EXAMPLE/20160718/US/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date,Signature=f58c63587c00d0c01122db06cf0b8a7376a2484b0572db3c562edddbd6d2fdb1', 'x-amz-date': '20160718T104632Z'} 
DEBUG: Processing request, please wait... 
DEBUG: get_hostname(test): s3.mylocalaws.com:9444/s3/test 
DEBUG: ConnMan.get(): creating new connection: http://s3.mylocalaws.com:9444/s3/test 

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 
    An unexpected error has occurred. 
    Please try reproducing the error using 
    the latest s3cmd code from the git master 
    branch found at: 
    https://github.com/s3tools/s3cmd 
    and have a look at the known issues list: 
    https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions 
    If the error persists, please report the 
    following lines (removing any private 
    info as necessary) to: 
    [email protected] 

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 

DEBUG: Unicodising '/usr/local/bin/s3cmd' using UTF-8 
DEBUG: Unicodising 'ls' using UTF-8 
DEBUG: Unicodising 's3://test' using UTF-8 
DEBUG: Unicodising '--debug' using UTF-8 
Invoked as: /usr/local/bin/s3cmd ls s3://test --debug 
Problem: UnboundLocalError: local variable 'conn' referenced before assignment 
S3cmd: 1.6.0 
python: 2.7.11 (default, Jan 22 2016, 08:29:18) 
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)] 
environment LANG=de_DE.UTF-8 

Traceback (most recent call last): 
    File "/usr/local/bin/s3cmd", line 2805, in <module> 
    rc = main() 
    File "/usr/local/bin/s3cmd", line 2713, in main 
    rc = cmd_func(args) 
    File "/usr/local/bin/s3cmd", line 120, in cmd_ls 
    subcmd_bucket_list(s3, uri) 
    File "/usr/local/bin/s3cmd", line 153, in subcmd_bucket_list 
    response = s3.bucket_list(bucket, prefix = prefix) 
    File "/usr/local/lib/python2.7/site-packages/S3/S3.py", line 293, in bucket_list 
    for dirs, objects in self.bucket_list_streaming(bucket, prefix, recursive, uri_params): 
    File "/usr/local/lib/python2.7/site-packages/S3/S3.py", line 320, in bucket_list_streaming 
    response = self.bucket_list_noparse(bucket, prefix, recursive, uri_params) 
    File "/usr/local/lib/python2.7/site-packages/S3/S3.py", line 339, in bucket_list_noparse 
    response = self.send_request(request) 
    File "/usr/local/lib/python2.7/site-packages/S3/S3.py", line 1084, in send_request 
    conn.counter = ConnMan.conn_max_counter 
UnboundLocalError: local variable 'conn' referenced before assignment 

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 
    An unexpected error has occurred. 
    Please try reproducing the error using 
    the latest s3cmd code from the git master 
    branch found at: 
    https://github.com/s3tools/s3cmd 
    and have a look at the known issues list: 
    https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions 
    If the error persists, please report the 
    above lines (removing any private 
    info as necessary) to: 
    [email protected] 
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 
+0

questo ancora non funziona -.- –

risposta

3

s3cmd al api forze momento di essere al /. D'altra parte, s3ninja non consente di impostarlo su "/". Ho proposto la funzionalità (PR 781) su s3cmd da gestire - è possibile correggere s3cmd da solo.

Un'altra soluzione sarebbe quella di impostare alcuni proxy, ad es. nginx />/s3

+0

Hai guadagnato quella taglia di sicuro! Questo è esattamente quello che ho fatto qualche giorno fa (Proxy). Aggiungerò presto un'altra risposta con la configurazione del proxy. –

1

Questa è la mia configurazione proxy nginx che funziona perfettamente con s3cmd e S3 Ninja.

server { 
     listen 80; 
     server_name s3.eu-central-1.amazonaws.com; 

     location/{ 
      proxy_pass   http://127.0.0.1:9444/s3/; 
     } 
    } 

con la seguente configurazione host:

host_base = s3.eu-central-1.amazonaws.com 
host_bucket = s3.eu-central-1.amazonaws.com 

Ed è necessario aggiungere una voce a/etc/hosts:

127.0.0.1 s3.eu-central-1.amazonaws.com 

Questo funziona anche con un proxy inverso e https.

0

S3cmd utilizza endpoint AWS e non è pronto a funzionare con interfacce compatibili S3. Fallisce quando si prova "get bucket location". .

--bucket-location=BUCKET_LOCATION 

Datacentre per creare secchio partire da ora i datacenter sono: US (default), UE, USA-ovest-1, e ap- sud-est-1

http://linux.die.net/man/1/s3cmd