Python che ha problemi ad accedere al microfono USB usando Gstreamer per eseguire il riconoscimento vocale con Pocketsphinx su un Raspberry Pi

Question

Quindi Python si comporta come se non fosse in grado di ascoltare NULLA dal mio microfono.

Ecco il problema. Ho uno script Python (2.7) che supponiamo utilizzi Gstreamer per accedere al mio microfono e fare riconoscimento vocale per me tramite Pocketsphinx. Sto usando Pulse Audio e il mio dispositivo è un Raspberry Pi. Il mio microfono è Playstation 3 Eye.

Ora, ho già fatto in modo che le tasche di ixx_continuous funzionino correttamente e riconosca le parole che ho definito nei miei file .dict e .lm. La precisione è all'incirca dell'85-90% accurata dopo un paio di prove che ho avuto. Quindi, dal mio punto di vista, so che il mio microfono sta rilevando il suono normalmente tramite l'audio di pocketphinx + pulse.

FYI Ho eseguito il seguente:

pocketsphinx_continuous -lm /home/pi/dev/scarlettPi/config/speech/lm/scarlett.lm -dict /home/pi/dev/scarlettPi/config/speech/dict/scarlett.dic -hmm /home/pi/dev/scarlettPi/config/speech/model/hmm/en_US/hub4wsj_sc_8k -silprob 0.1 -wip 1e-4 -bestpath 0 

Nel mio codice python che sto cercando di fare la stessa cosa, ma sto usando gstreamer per accedere al microfono in pitone. (Nota: Sono un po 'nuova per Python)

Ecco il mio codice (Grazie Josip Lisec per me ottenere questo lontano):

import pi 
from pi.becore import ScarlettConfig 
from recorder import Recorder 
from brain import Brain 

import os 
import json 
import tempfile 
#import sys 

import pygtk 
pygtk.require('2.0') 
import gtk 
import gobject 
import pygst 
pygst.require('0.10') 
gobject.threads_init() 
import gst 

scarlett_config=ScarlettConfig() 

class Listener: 
    def __init__(self, gobject, gst): 
    self.failed = 0 

    self.pipeline = gst.parse_launch(' ! '.join(['pulsesrc', 
               'audioconvert', 
               'audioresample', 
               'vader name=vader auto-threshold=true', 
               'pocketsphinx lm=' + scarlett_config.get('LM') + ' dict=' + scarlett_config.get('DICT') + ' hmm=' + scarlett_config.get('HMM') + ' name=listener', 
               'fakesink'])) 
    listener = self.pipeline.get_by_name('listener') 
    listener.connect('result', self.__result__) 
    listener.set_property('configured', True) 
    print "KEYWORDS WE'RE LOOKING FOR: " + scarlett_config.get('ourkeywords') 

    bus = self.pipeline.get_bus() 
    bus.add_signal_watch() 
    bus.connect('message::application', self.__application_message__) 
    self.pipeline.set_state(gst.STATE_PLAYING) 

    def result(self, hyp, uttid): 
    if hyp in scarlett_config.get('ourkeywords'): 
     self.failed = 0 
     self.listen() 
    else: 
     self.failed += 1 
     if self.failed > 4: 
     pi.speak("" + scarlett_config.get('scarlett_owner') + ", if you need me, just say my name.") 
     self.failed = 0 

    def listen(self): 
    self.pipeline.set_state(gst.STATE_PAUSED) 
    pi.play('pi-listening') 
    Recorder(self) 

    def cancel_listening(self): 
    pi.play('pi-cancel') 
    self.pipeline.set_state(gst.STATE_PLAYING) 

    # question - sound recording 
    def answer(self, question): 
    pi.play('pi-cancel') 

    print " * Contacting Google" 
    destf = tempfile.mktemp(suffix='piresult') 
    os.system('wget --post-file %s --user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7" --header="Content-Type: audio/x-flac; rate=16000" -O %s -q "https://www.google.com/speech-api/v1/recognize?client=chromium&lang=en-US"' % (question, destf)) 
    #os.system("speech2text %s > %s" % (question, destf)) 
    b = open(destf) 
    result = b.read() 
    b.close() 

    os.unlink(question) 
    os.unlink(destf) 

    if len(result) == 0: 
     print " * nop" 
     pi.play('pi-cancel') 
    else: 
     brain = Brain(json.loads(result)) 
     if brain.think() == False: 
     print " * nop2" 
     pi.play('pi-cancel') 

    self.pipeline.set_state(gst.STATE_PLAYING) 

    def __result__(self, listener, text, uttid): 
    struct = gst.Structure('result') 
    struct.set_value('hyp', text) 
    struct.set_value('uttid', uttid) 
    listener.post_message(gst.message_new_application(listener, struct)) 

    def __application_message__(self, bus, msg): 
    msgtype = msg.structure.get_name() 
    if msgtype == 'result': 
     self.result(msg.structure['hyp'], msg.structure['uttid']) 

L'applicazione è supponiamo di corrispondere sulla parola chiave "Scarlett" quindi eseguire un'azione dopo.

Quando eseguo la mia applicazione, ottengo il seguente output:

pi@scarlettpi ~/dev/scarlettPi/scripts/pi/bin $ ./pi 
/usr/lib/python2.7/dist-packages/gtk-2.0/gtk/__init__.py:57: GtkWarning: could not open display 
    warnings.warn(str(e), _gtk.Warning) 
INFO: cmd_ln.c(691): Parsing command line: 
gst-pocketsphinx \ 
    -samprate 8000 \ 
    -cmn prior \ 
    -fwdflat no \ 
    -bestpath no \ 
    -maxhmmpf 2000 \ 
    -maxwpf 20 

Current configuration: 
[NAME]  [DEFLT]  [VALUE] 
-agc  none  none 
-agcthresh 2.0  2.000000e+00 
-alpha  0.97  9.700000e-01 
-ascale  20.0  2.000000e+01 
-aw  1  1 
-backtrace no  no 
-beam  1e-48  1.000000e-48 
-bestpath no  no 
-bestpathlw 9.5  9.500000e+00 
-bghist  no  no 
-ceplen  13  13 
-cmn  current  prior 
-cmninit 8.0  8.0 
-compallsen no  no 
-debug    0 
-dict    
-dictcase no  no 
-dither  no  no 
-doublebw no  no 
-ds  1  1 
-fdict    
-feat  1s_c_d_dd 1s_c_d_dd 
-featparams   
-fillprob 1e-8  1.000000e-08 
-frate  100  100 
-fsg     
-fsgusealtpron yes  yes 
-fsgusefiller yes  yes 
-fwdflat yes  no 
-fwdflatbeam 1e-64  1.000000e-64 
-fwdflatefwid 4  4 
-fwdflatlw 8.5  8.500000e+00 
-fwdflatsfwin 25  25 
-fwdflatwbeam 7e-29  7.000000e-29 
-fwdtree yes  yes 
-hmm     
-input_endian little  little 
-jsgf    
-kdmaxbbi -1  -1 
-kdmaxdepth 0  0 
-kdtree    
-latsize 5000  5000 
-lda     
-ldadim  0  0 
-lextreedump 0  0 
-lifter  0  0 
-lm    
-lmctl    
-lmname  default  default 
-logbase 1.0001  1.000100e+00 
-logfn    
-logspec no  no 
-lowerf  133.33334 1.333333e+02 
-lpbeam  1e-40  1.000000e-40 
-lponlybeam 7e-29  7.000000e-29 
-lw  6.5  6.500000e+00 
-maxhmmpf -1  2000 
-maxnewoov 20  20 
-maxwpf  -1  20 
-mdef    
-mean    
-mfclogdir   
-min_endfr 0  0 
-mixw    
-mixwfloor 0.0000001 1.000000e-07 
-mllr    
-mmap  yes  yes 
-ncep  13  13 
-nfft  512  512 
-nfilt  40  40 
-nwpen  1.0  1.000000e+00 
-pbeam  1e-48  1.000000e-48 
-pip  1.0  1.000000e+00 
-pl_beam 1e-10  1.000000e-10 
-pl_pbeam 1e-5  1.000000e-05 
-pl_window 0  0 
-rawlogdir   
-remove_dc no  no 
-round_filters yes  yes 
-samprate 16000  8.000000e+03 
-seed  -1  -1 
-sendump    
-senlogdir   
-senmgau    
-silprob 0.1  1.000000e-01 
-smoothspec no  no 
-svspec    
-tmat    
-tmatfloor 0.0001  1.000000e-04 
-topn  4  4 
-topn_beam 0  0 
-toprule    
-transform legacy  legacy 
-unit_area yes  yes 
-upperf  6855.4976 6.855498e+03 
-usewdphones no  no 
-uw  1.0  1.000000e+00 
-var     
-varfloor 0.0001  1.000000e-04 
-varnorm no  no 
-verbose no  no 
-warp_params    
-warp_type inverse_linear inverse_linear 
-wbeam  7e-29  7.000000e-29 
-wip  1e-4  1.000000e-04 
-wlen  0.025625 2.562500e-02 

INFO: cmd_ln.c(691): Parsing command line: 
\ 
    -nfilt 20 \ 
    -lowerf 1 \ 
    -upperf 4000 \ 
    -wlen 0.025 \ 
    -transform dct \ 
    -round_filters no \ 
    -remove_dc yes \ 
    -svspec 0-12/13-25/26-38 \ 
    -feat 1s_c_d_dd \ 
    -agc none \ 
    -cmn current \ 
    -cmninit 56,-3,1 \ 
    -varnorm no 

Current configuration: 
[NAME]  [DEFLT]  [VALUE] 
-agc  none  none 
-agcthresh 2.0  2.000000e+00 
-alpha  0.97  9.700000e-01 
-ceplen  13  13 
-cmn  current  current 
-cmninit 8.0  56,-3,1 
-dither  no  no 
-doublebw no  no 
-feat  1s_c_d_dd 1s_c_d_dd 
-frate  100  100 
-input_endian little  little 
-lda     
-ldadim  0  0 
-lifter  0  0 
-logspec no  no 
-lowerf  133.33334 1.000000e+00 
-ncep  13  13 
-nfft  512  512 
-nfilt  40  20 
-remove_dc no  yes 
-round_filters yes  no 
-samprate 16000  8.000000e+03 
-seed  -1  -1 
-smoothspec no  no 
-svspec    0-12/13-25/26-38 
-transform legacy  dct 
-unit_area yes  yes 
-upperf  6855.4976 4.000000e+03 
-varnorm no  no 
-verbose no  no 
-warp_params    
-warp_type inverse_linear inverse_linear 
-wlen  0.025625 2.500000e-02 

INFO: acmod.c(246): Parsed model-specific feature parameters from /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/feat.params 
INFO: feat.c(713): Initializing feature stream to type: '1s_c_d_dd', ceplen=13, CMN='current', VARNORM='no', AGC='none' 
INFO: cmn.c(142): mean[0]= 12.00, mean[1..12]= 0.0 
INFO: acmod.c(167): Using subvector specification 0-12/13-25/26-38 
INFO: mdef.c(517): Reading model definition: /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/mdef 
INFO: mdef.c(528): Found byte-order mark BMDF, assuming this is a binary mdef file 
INFO: bin_mdef.c(336): Reading binary model definition: /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/mdef 
INFO: bin_mdef.c(513): 50 CI-phone, 143047 CD-phone, 3 emitstate/phone, 150 CI-sen, 5150 Sen, 27135 Sen-Seq 
INFO: tmat.c(205): Reading HMM transition probability matrices: /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/transition_matrices 
INFO: acmod.c(121): Attempting to use SCHMM computation module 
INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/means 
INFO: ms_gauden.c(292): 1 codebook, 3 feature, size: 
INFO: ms_gauden.c(294): 256x13 
INFO: ms_gauden.c(294): 256x13 
INFO: ms_gauden.c(294): 256x13 
INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/variances 
INFO: ms_gauden.c(292): 1 codebook, 3 feature, size: 
INFO: ms_gauden.c(294): 256x13 
INFO: ms_gauden.c(294): 256x13 
INFO: ms_gauden.c(294): 256x13 
INFO: ms_gauden.c(354): 0 variance values floored 
INFO: s2_semi_mgau.c(903): Loading senones from dump file /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/sendump 
INFO: s2_semi_mgau.c(927): BEGIN FILE FORMAT DESCRIPTION 
INFO: s2_semi_mgau.c(1022): Using memory-mapped I/O for senones 
INFO: s2_semi_mgau.c(1296): Maximum top-N: 4 Top-N beams: 0 0 0 
INFO: dict.c(317): Allocating 4120 * 20 bytes (80 KiB) for word entries 
INFO: dict.c(332): Reading main dictionary: /home/pi/dev/scarlettPi/config/speech/dict/scarlett.dic 
INFO: dict.c(211): Allocated 0 KiB for strings, 0 KiB for phones 
INFO: dict.c(335): 13 words read 
INFO: dict.c(341): Reading filler dictionary: /usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/noisedict 
INFO: dict.c(211): Allocated 0 KiB for strings, 0 KiB for phones 
INFO: dict.c(344): 11 words read 
INFO: dict2pid.c(396): Building PID tables for dictionary 
INFO: dict2pid.c(404): Allocating 50^3 * 2 bytes (244 KiB) for word-initial triphones 
INFO: dict2pid.c(131): Allocated 30200 bytes (29 KiB) for word-final triphones 
INFO: dict2pid.c(195): Allocated 30200 bytes (29 KiB) for single-phone word triphones 
INFO: ngram_model_arpa.c(477): ngrams 1=12, 2=18, 3=17 
INFO: ngram_model_arpa.c(135): Reading unigrams 
INFO: ngram_model_arpa.c(516):  12 = #unigrams created 
INFO: ngram_model_arpa.c(195): Reading bigrams 
INFO: ngram_model_arpa.c(533):  18 = #bigrams created 
INFO: ngram_model_arpa.c(534):  3 = #prob2 entries 
INFO: ngram_model_arpa.c(542):  3 = #bo_wt2 entries 
INFO: ngram_model_arpa.c(292): Reading trigrams 
INFO: ngram_model_arpa.c(555):  17 = #trigrams created 
INFO: ngram_model_arpa.c(556):  2 = #prob3 entries 
INFO: ngram_search_fwdtree.c(99): 12 unique initial diphones 
INFO: ngram_search_fwdtree.c(147): 0 root, 0 non-root channels, 12 single-phone words 
INFO: ngram_search_fwdtree.c(186): Creating search tree 
INFO: ngram_search_fwdtree.c(191): before: 0 root, 0 non-root channels, 12 single-phone words 
INFO: ngram_search_fwdtree.c(326): after: max nonroot chan increased to 152 
INFO: ngram_search_fwdtree.c(338): after: 12 root, 24 non-root channels, 11 single-phone words 
KEYWORDS WE'RE LOOKING FOR: [ 'scarlett', 'SCARLETT' ]  

ma non riesce a corrispondere su qualsiasi cosa. Penso quasi che Python non riesca a sentire nulla dal microfono, non ci sono nemmeno tentativi di riconoscere qualcosa. In pocketphinx_continuo di solito stampa uno stato PRONTO quando è pronto per iniziare ad ascoltare ... Mi aspetto lo stesso in python?

Qui sono i miei pacchetti python:

pi@scarlettpi ~/dev/scarlettPi/scripts/pi/bin $ dpkg -l | grep -i python 
ii idle         2.7.3-4        all   IDE for Python using Tkinter (default version) 
ii idle-python2.7      2.7.3-6        all   IDE for Python (v2.7) using Tkinter 
rc idle3         3.2.3-6        all   IDE for Python using Tkinter (default version) 
ii libpyside1.1:armhf     1.1.1-3        armhf  Python bindings for Qt 4 (base files) 
ii libpython2.6       2.6.8-1.1       armhf  Shared Python runtime library (version 2.6) 
ii libpython2.7       2.7.3-6        armhf  Shared Python runtime library (version 2.7) 
ii libshiboken1.1:armhf     1.1.1-1        armhf  CPython bindings generator for C++ libraries - shared library 
ii python        2.7.3-4        all   interactive high-level object-oriented language (default version) 
ii python-alsaaudio      0.5+svn36-1       armhf  Alsa bindings for Python 
ii python-cairo       1.8.8-1        armhf  Python bindings for the Cairo vector graphics library 
ii python-dbg       2.7.3-4        all   debug build of the Python Interpreter (version 2.7) 
ii python-dbus       1.1.1-1        armhf  simple interprocess messaging system (Python interface) 
ii python-dbus-dev      1.1.1-1        all   main loop integration development files for python-dbus 
ii python-dev       2.7.3-4        all   header files and a static library for Python (default) 
ii python-gi        3.2.2-2        armhf  Python 2.x bindings for gobject-introspection libraries 
ii python-gi-dbg       3.2.2-2        armhf  Python bindings for the GObject library (debug extension) 
ii python-gi-dev       3.2.2-2        all   development headers for GObject Python bindings 
ii python-gobject      3.2.2-2        all   Python 2.x bindings for GObject - transitional package 
ii python-gobject-2      2.28.6-10       armhf  deprecated static Python bindings for the GObject library 
ii python-gobject-2-dbg     2.28.6-10       armhf  deprecated static Python bindings for the GObject library (debug extension) 
ii python-gobject-2-dev     2.28.6-10       all   development headers for the static GObject Python bindings 
ii python-gobject-dbg     3.2.2-2        all   Python 2.x debugging modules for GObject - transitional package 
ii python-gobject-dev     3.2.2-2        all   Python 2.x development headers for GObject - transitional package 
ii python-gst0.10      0.10.22-3       armhf  generic media-playing framework (Python bindings) 
ii python-gst0.10-dbg     0.10.22-3       armhf  generic media-playing framework (Python debug bindings) 
ii python-gst0.10-dev     0.10.22-3       armhf  generic media-playing framework (Python bindings) 
ii python-gst0.10-rtsp     0.10.8-3        armhf  GStreamer RTSP server plugin (Python bindings) 
ii python-gtk2       2.24.0-3        armhf  Python bindings for the GTK+ widget set 
ii python-iplib       1.1-3        all   Python library to convert amongst many different IPv4 notations 
ii python-libxml2      2.8.0+dfsg1-7+nmu1     armhf  Python bindings for the GNOME XML library 
ii python-minimal      2.7.3-4        all   minimal subset of the Python language (default version) 
ii python-numpy       1:1.6.2-1.2       armhf  Numerical Python adds a fast array facility to the Python language 
ii python-pexpect      2.4-1        all   Python module for automating interactive applications 
ii python-pip       1.1-3        all   alternative Python package installer 
ii python-pkg-resources     0.6.24-1        all   Package Discovery and Resource Access using pkg_resources 
ii python-pyalsa       1.0.25-1        armhf  Official ALSA Python binding library 
ii python-pyside       1.1.1-3        all   Python bindings for Qt4 (big metapackage) 
ii python-pyside.phonon     1.1.1-3        armhf  Qt 4 Phonon module - Python bindings 
ii python-pyside.qtcore     1.1.1-3        armhf  Qt 4 core module - Python bindings 
ii python-pyside.qtdeclarative   1.1.1-3        armhf  Qt 4 Declarative module - Python bindings 
ii python-pyside.qtgui     1.1.1-3        armhf  Qt 4 GUI module - Python bindings 
ii python-pyside.qthelp     1.1.1-3        armhf  Qt 4 help module - Python bindings 
ii python-pyside.qtnetwork    1.1.1-3        armhf  Qt 4 network module - Python bindings 
ii python-pyside.qtopengl    1.1.1-3        armhf  Qt 4 OpenGL module - Python bindings 
ii python-pyside.qtscript    1.1.1-3        armhf  Qt 4 script module - Python bindings 
ii python-pyside.qtsql     1.1.1-3        armhf  Qt 4 SQL module - Python bindings 
ii python-pyside.qtsvg     1.1.1-3        armhf  Qt 4 SVG module - Python bindings 
ii python-pyside.qttest     1.1.1-3        armhf  Qt 4 test module - Python bindings 
ii python-pyside.qtuitools    1.1.1-3        armhf  Qt 4 UI tools module - Python bindings 
ii python-pyside.qtwebkit    1.1.1-3        armhf  Qt 4 WebKit module - Python bindings 
ii python-pyside.qtxml     1.1.1-3        armhf  Qt 4 XML module - Python bindings 
ii python-rpi.gpio      0.5.3a-1        armhf  Python GPIO module for Raspberry Pi 
ii python-setuptools      0.6.24-1        all   Python Distutils Enhancements (setuptools compatibility) 
ii python-simplejson      2.5.2-1        armhf  simple, fast, extensible JSON encoder/decoder for Python 
ii python-support      1.0.15        all   automated rebuilding support for Python modules 
ii python-tk        2.7.3-1        armhf  Tkinter - Writing Tk applications with Python 
ii python-yaml       3.10-4        armhf  YAML parser and emitter for Python 
ii python-yaml-dbg      3.10-4        armhf  YAML parser and emitter for Python (debug build) 
ii python2.6        2.6.8-1.1       armhf  Interactive high-level object-oriented language (version 2.6) 
ii python2.6-minimal      2.6.8-1.1       armhf  Minimal subset of the Python language (version 2.6) 
ii python2.7        2.7.3-6        armhf  Interactive high-level object-oriented language (version 2.7) 
ii python2.7-dbg       2.7.3-6        armhf  Debug Build of the Python Interpreter (version 2.7) 
ii python2.7-dev       2.7.3-6        armhf  Header files and a static library for Python (v2.7) 
ii python2.7-minimal      2.7.3-6        armhf  Minimal subset of the Python language (version 2.7) 
pi@scarlettpi ~/dev/scarlettPi/scripts/pi/bin $ 

anche solo per confermare che pocketsphinx sia rispettato correttamente contro i libaries giusti:

pi@scarlettpi ~ $ ldd /usr/local/bin/pocketsphinx_continuous 
    /usr/lib/arm-linux-gnueabihf/libcofi_rpi.so (0xb6f9b000) 
    libpocketsphinx.so.1 => /usr/local/lib/libpocketsphinx.so.1 (0xb6f5a000) 
    libsphinxad.so.0 => /usr/local/lib/libsphinxad.so.0 (0xb6f4e000) 
    libsphinxbase.so.1 => /usr/local/lib/libsphinxbase.so.1 (0xb6f07000) 
    libpulse.so.0 => /usr/lib/arm-linux-gnueabihf/libpulse.so.0 (0xb6ea8000) 
    libpulse-simple.so.0 => /usr/lib/arm-linux-gnueabihf/libpulse-simple.so.0 (0xb6e9c000) 
    libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0xb6e7d000) 
    libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0xb6e0c000) 
    libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0xb6cdd000) 
    libjson.so.0 => /lib/arm-linux-gnueabihf/libjson.so.0 (0xb6ccd000) 
    libpulsecommon-2.0.so => /usr/lib/arm-linux-gnueabihf/pulseaudio/libpulsecommon-2.0.so (0xb6c6b000) 
    libdbus-1.so.3 => /lib/arm-linux-gnueabihf/libdbus-1.so.3 (0xb6c29000) 
    libcap.so.2 => /lib/arm-linux-gnueabihf/libcap.so.2 (0xb6c1e000) 
    librt.so.1 => /lib/arm-linux-gnueabihf/librt.so.1 (0xb6c0f000) 
    libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0xb6c04000) 
    libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0xb6bdb000) 
    /lib/ld-linux-armhf.so.3 (0xb6fa8000) 
    libX11-xcb.so.1 => /usr/lib/arm-linux-gnueabihf/libX11-xcb.so.1 (0xb6bd2000) 
    libX11.so.6 => /usr/lib/arm-linux-gnueabihf/libX11.so.6 (0xb6abe000) 
    libxcb.so.1 => /usr/lib/arm-linux-gnueabihf/libxcb.so.1 (0xb6a9f000) 
    libICE.so.6 => /usr/lib/arm-linux-gnueabihf/libICE.so.6 (0xb6a82000) 
    libSM.so.6 => /usr/lib/arm-linux-gnueabihf/libSM.so.6 (0xb6a73000) 
    libXtst.so.6 => /usr/lib/arm-linux-gnueabihf/libXtst.so.6 (0xb6a67000) 
    libwrap.so.0 => /lib/arm-linux-gnueabihf/libwrap.so.0 (0xb6a57000) 
    libsndfile.so.1 => /usr/lib/arm-linux-gnueabihf/libsndfile.so.1 (0xb69ee000) 
    libasyncns.so.0 => /usr/lib/arm-linux-gnueabihf/libasyncns.so.0 (0xb69e2000) 
    libattr.so.1 => /lib/arm-linux-gnueabihf/libattr.so.1 (0xb69d4000) 
    libXau.so.6 => /usr/lib/arm-linux-gnueabihf/libXau.so.6 (0xb69ca000) 
    libXdmcp.so.6 => /usr/lib/arm-linux-gnueabihf/libXdmcp.so.6 (0xb69be000) 
    libuuid.so.1 => /lib/arm-linux-gnueabihf/libuuid.so.1 (0xb69b1000) 
    libXext.so.6 => /usr/lib/arm-linux-gnueabihf/libXext.so.6 (0xb699b000) 
    libXi.so.6 => /usr/lib/arm-linux-gnueabihf/libXi.so.6 (0xb6986000) 
    libnsl.so.1 => /lib/arm-linux-gnueabihf/libnsl.so.1 (0xb696a000) 
    libFLAC.so.8 => /usr/lib/arm-linux-gnueabihf/libFLAC.so.8 (0xb691f000) 
    libvorbisenc.so.2 => /usr/lib/arm-linux-gnueabihf/libvorbisenc.so.2 (0xb67b2000) 
    libvorbis.so.0 => /usr/lib/arm-linux-gnueabihf/libvorbis.so.0 (0xb6782000) 
    libogg.so.0 => /usr/lib/arm-linux-gnueabihf/libogg.so.0 (0xb6775000) 
    libresolv.so.2 => /lib/arm-linux-gnueabihf/libresolv.so.2 (0xb6761000) 
pi@scarlettpi ~ $ 

E se avete bisogno di vedere tutte le informazioni sul mio microfono (ps3 eye):

dovuto gettare questo in pastebin, ha esaurito la stanza in questo post.

http://pastebin.com/gSDZwRHc

Qualcuno ha qualche idea perché questo non sta funzionando? Per favore fatemi sapere se la mia domanda ha bisogno di qualche chiarimento o se posso fornire ulteriori informazioni per aiutare con il debug.

Grazie.

Answer 1

Così finalmente ho fatto funzionare questo tizio.

paio di cose fondamentali di cui avevo bisogno per rendersi conto:

1. Anche se si sta utilizzando Pulseaudio sul vostro Raspberry Pi, a patto che Alsa è ancora installato si sono ancora in grado di usarlo. (Questo potrebbe sembrare un gioco da ragazzi per gli altri, ma onestamente non mi ero reso conto che potevo ancora usare entrambi allo stesso tempo) Suggerimento via (syb0rg).

2. Quando si tratta di invio di grandi quantità di dati audio grezzi (.wav formato nel mio caso) a pocketsphinx via Gstreamer, (queues) sono tuo amico.

Dopo fare in giro con gst-launch-0.10 sulla riga di comando per un po 'mi sono imbattuto in qualcosa che in realtà ha funzionato:

gst-launch-0.10 alsasrc device=hw:1 ! queue ! audioconvert ! audioresample ! queue ! vader name=vader auto-threshold=true ! pocketsphinx lm=/home/pi/dev/scarlettPi/config/speech/lm/scarlett.lm dict=/home/pi/dev/scarlettPi/config/speech/dict/scarlett.dic hmm=/usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k name=listener ! fakesink dump=1 

Quindi, cosa sta succedendo qui?

• Gstreamer sta ascoltando il dispositivo hw: 1 (che è il mio dispositivo Ps3 Eye USB). Questo dispositivo potrebbe variare, si può determinare eseguendo:

pi@scarlettpi ~ $ pacmd dump 
Welcome to PulseAudio! Use "help" for usage information. 

.... 

load-module module-alsa-card device_id="0" name="platform-bcm2835_AUD0.0" 

CARD_NAME = namereg_fail "alsa_card.platform-bcm2835_AUD0.0" = false TSched = yes fixed_latency_range = no ignore_dB = no deferred_volume = yes card_properties = "modulo-udev-detect.discovered = 1"

load-module module-udev-detect 

load-module module-bluetooth-discover 

load-module module-esound-protocol-unix 

load-module module-native-protocol-unix 

load-module module-gconf 

load-module module-default-device-restore 

load-module module-rescue-streams 

load-module module-always-sink 

load-module module-intended-roles 

load-module module-console-kit 

load-module module-systemd-login 

load-module module-position-event-sounds 

load-module module-role-cork 

load-module module-filter-heuristics 

load-module module-filter-apply 

load-module module-dbus-protocol 

load-module module-switch-on-port-available 

load-module module-cli-protocol-unix 

load-module module-alsa-card device_id="1" name="usb-OmniVision_Technologies__Inc._USB_Camera-B4.09.24.1-01-CameraB409241" card_name="alsa_card.usb-OmniVision_Technologies__Inc._USB_Camera-B4.09.24.1-01-CameraB409241" namereg_fail=false tsched=yes fixed_latency_range=no ignore_dB=no 

deferred_volume = yes card_properties = "modulo-udev-detect.discovered = 1"

.... 

La linea importante da notare è:

load-module module-alsa-card device_id="1" name="usb-OmniVision_Technologies__Inc._USB_Camera-B4.09.24.1-01-CameraB409241" card_name="alsa_card.usb-OmniVision_Technologies__Inc._USB_Camera-B4.09.24.1-01-CameraB409241" namereg_fail=false tsched=yes fixed_latency_range=no ignore_dB=no deferred_volume=yes card_properties="module-udev-detect.discovered=1" 

Quello è il mio Playstation 3 Eye, e questo è in device_id = 1. Quindi hw: 1

• I dati audio provenienti dall'occhio ps3 ottiene resampled e aggiunto a una coda gstreamer e deve passare attraverso un elemento (vader) prima di passare a pocketsphinx. Passando l'audio attraverso l'elemento vader con l'auto-threshold = true flag attivo, gstreamer può determinare il livello di rumore di fondo, che può essere importante se si dispone di una scheda audio scadente o di un microfono a campo lontano. Questo è il modo in cui l'elemento pocketphinx saprà quando un enunciato inizia e finisce.

• Aggiungere i normali argomenti pocketphix alla pipeline che abbiamo già determinato (here).

• Passa tutto a un falso perché non abbiamo bisogno di sentire niente adesso, abbiamo solo bisogno di pocketphinx per ascoltare tutto. Il flag dump = 1 ci fornisce più informazioni di debug per vedere cosa viene elaborato/se l'audio viene accettato a tutti.

** Dopo aver ottenuto che per eseguire con successo, il nuovo codice Python assomiglia a questo: **

self.pipeline = gst.parse_launch(' ! '.join(['alsasrc device=' + scarlett_config.gimmie('audio_input_device'), 
              'queue', 
              'audioconvert', 
              'audioresample', 
              'queue', 
              'vader name=vader auto-threshold=true', 
              'pocketsphinx lm=' + scarlett_config.gimmie('LM') + ' dict=' + scarlett_config.gimmie('DICT') + ' hmm=' + scarlett_config.gimmie('HMM') + ' name=listener', 
              'fakesink dump=1'])) 

Spero che questo aiuti qualcuno.

NOTA: Si prega di scusarmi se il mio gasdotto Gstreamer utilizza elementi eccessivi. Sono abbastanza nuovo per Gstreamer, e sto aprendo a modi più efficienti per farlo.