La colonna data.table di divisione stringa produce NA

Question

Questa è la mia prima domanda su SO, quindi fammi sapere se può essere migliorata. Sto lavorando a un progetto di elaborazione del linguaggio naturale in R e sto cercando di creare un data.table che contenga casi di test. Qui, a costruire un gran esempio semplificato:

texts.dt <- data.table(string = c("one", 
            "two words", 
            "three words here", 
            "four useless words here", 
            "five useless meaningless words here", 
            "six useless meaningless words here just", 
            "seven useless meaningless words here just to", 
            "eigth useless meaningless words here just to fill", 
            "nine useless meaningless words here just to fill up", 
            "ten useless meaningless words here just to fill up space"), 
         word.count = 1:10, 
         stop.at.word = c(0, 1, 2, 2, 4, 3, 3, 6, 7, 5)) 

Ciò restituisce il data.table lavoreremo su:

              string word.count stop.at.word 
1:              one   1   0 
2:            two words   2   1 
3:           three words here   3   2 
4:         four useless words here   4   2 
5:      five useless meaningless words here   5   4 
6:     six useless meaningless words here just   6   3 
7:    seven useless meaningless words here just to   7   3 
8:  eigth useless meaningless words here just to fill   8   6 
9:  nine useless meaningless words here just to fill up   9   7 
10: ten useless meaningless words here just to fill up space   10   5 

In applicazione reale, i valori nella colonna stop.at.word sono determinati a caso (con un limite superiore = word.count - 1). Inoltre, le stringhe non sono ordinate per lunghezza ma ciò non dovrebbe fare la differenza.

Il codice dovrebbe aggiungere due colonne input e output, dove input contiene la stringa dalla posizione 1 fino a stop.at.word e output contiene la parola che segue (parola), in questo modo:

>desired_result 
                  string word.count stop.at.word          input 
    1:              one   1   0            
    2:            two words   2   1           two 
    3:           three words here   3   2         three words 
    4:         four useless words here   4   2        four useless 
    5:      five useless meaningless words here   5   4    five useless meaningless words 
    6:     six useless meaningless words here just   6   2         six useless 
    7:    seven useless meaningless words here just to   7   3     seven useless meaningless 
    8:  eigth useless meaningless words here just to fill   8   6 eigth useless meaningless words here just 
    9:  nine useless meaningless words here just to fill up   9   7 nine useless meaningless words here just to 
    10: ten useless meaningless words here just to fill up space   10   5   ten useless meaningless words here 
      output 
    1:    
    2:  words 
    3:  here 
    4:  words 
    5:  here 
    6: meaningless 
    7:  words 
    8:   to 
    9:  fill 
    10:  just 

Purtroppo quello che ho ottenere invece è questo:

             string word.count stop.at.word input output 
1:              one   1   0    
2:            two words   2   1 NA  NA 
3:           three words here   3   2 NA  NA 
4:         four useless words here   4   2 NA  NA 
5:      five useless meaningless words here   5   4 NA  NA 
6:     six useless meaningless words here just   6   3 NA  NA 
7:    seven useless meaningless words here just to   7   3 NA  NA 
8:  eigth useless meaningless words here just to fill   8   6 NA  NA 
9:  nine useless meaningless words here just to fill up   9   7 NA  NA 
10: ten useless meaningless words here just to fill up space   10   5 ten  NA 

Avviso i risultati incoerenti, con una stringa vuota nel braccio 1 e "dieci" restituiti nel braccio 10.

Ecco il codice che sto usando:

texts.dt[, c("input", "output") := .(
     substr(string, 
       1, 
       sapply(gregexpr(" ", string),"[", stop.at.word) - 1), 
     substr(string, 
       sapply(gregexpr(" ", string),"[", stop.at.word), 
       sapply(gregexpr(" ", string),"[", stop.at.word + 1) - 1) 
    )] 

Ho eseguito molti test e le istruzioni substr funzionano bene quando provo singole stringhe nella console, ma non riescono quando applicato alla data.table. Ho il sospetto che manchi qualcosa relativo all'ambito dell'ambito all'interno di data.table, ma non uso questo pacchetto da molto tempo, quindi sono abbastanza confuso.

Apprezzerei molto l'assistenza. Grazie in anticipo!

Answer 1

avrei probabilmente fare

texts.dt[stop.at.word > 0, c("input","output") := { 
    sp = strsplit(string, " ") 
    list( 
    mapply(function(p,n) paste(p[seq_len(n)], collapse = " "), sp, stop.at.word), 
    mapply(`[`, sp, stop.at.word+1L) 
) 
}] 

# partial result 
head(texts.dt, 4) 

        string word.count stop.at.word  input output 
1:      one   1   0   NA  NA 
2:    two words   2   1   two words 
3:  three words here   3   2 three words here 
4: four useless words here   4   2 four useless words 

---

alternativa:

library(stringi) 
texts.dt[stop.at.word > 0, c("input","output") := { 
    patt = paste0("((\\w+){", stop.at.word-1, "}\\w+) (.*)") 
    m = stri_match(string, regex = patt) 
    list(m[, 2], m[, 4]) 
}] 

Answer 2

Un'alternativa alla soluzione @ di Frank mapply sta usando by = 1:nrow(texts.dt) con strsplit e paste:

library(data.table) 
texts.dt[, `:=` (input = paste(strsplit(string, ' ')[[1]][1:stop.at.word][stop.at.word>0], 
           collapse = " "), 
       output = strsplit(string, ' ')[[1]][stop.at.word + 1]), 
     by = 1:nrow(texts.dt)] 

che dà:

> texts.dt 
                 string word.count stop.at.word          input output 
1:              one   1   0            one 
2:            two words   2   1           two words 
3:           three words here   3   2         three words here 
4:         four useless words here   4   2        four useless words 
5:      five useless meaningless words here   5   4    five useless meaningless words here 
6:     six useless meaningless words here just   6   3      six useless meaningless words 
7:    seven useless meaningless words here just to   7   3     seven useless meaningless words 
8:  eigth useless meaningless words here just to fill   8   6 eigth useless meaningless words here just  to 
9:  nine useless meaningless words here just to fill up   9   7 nine useless meaningless words here just to fill 
10: ten useless meaningless words here just to fill up space   10   5   ten useless meaningless words here just 

Invece di utilizzare [[1]] si potrebbe anche avvolgere il strsplit in unlist come segue: unlist(strsplit(string, ' ')) (invece di strsplit(string, ' ')[[1]]). Questo ti darà lo stesso risultato.altri

---

due opzioni:

1) con il pacchetto stringi:

library(stringi) 
texts.dt[, `:=`(input = paste(stri_extract_all_words(string[stop.at.word>0], 
                simplify = TRUE)[1:stop.at.word], 
           collapse = " "), 
       output = stri_extract_all_words(string[stop.at.word>0], 
               simplify = TRUE)[stop.at.word+1]), 
     1:nrow(texts.dt)] 

2) o un adattamento da this answer:

texts.dt[stop.at.word>0, 
     c('input','output') := tstrsplit(string, 
              split = paste0("(?=(?>\\s+\\S*){", 
                 word.count - stop.at.word, 
                 "}$)\\s"), 
              perl = TRUE) 
     ][, output := sub('(\\w+).*','\\1',output)] 

.515.053.691,36321 milioni

che entrambi danno:

> texts.dt 
                 string word.count stop.at.word          input output 
1:              one   1   0           NA  NA 
2:            two words   2   1           two words 
3:           three words here   3   2         three words here 
4:         four useless words here   4   2        four useless words 
5:      five useless meaningless words here   5   4    five useless meaningless words here 
6:     six useless meaningless words here just   6   3      six useless meaningless words 
7:    seven useless meaningless words here just to   7   3     seven useless meaningless words 
8:  eigth useless meaningless words here just to fill   8   6 eigth useless meaningless words here just  to 
9:  nine useless meaningless words here just to fill up   9   7 nine useless meaningless words here just to fill 
10: ten useless meaningless words here just to fill up space   10   5   ten useless meaningless words here just 

Answer 3

dt[, `:=`(input = sub(paste0('((\\s*\\w+){', stop.at.word, '}).*'), '\\1', string), 
      output = sub(paste0('(\\s*\\w+){', stop.at.word, '}\\s*(\\w+).*'), '\\2', string)) 
    , by = stop.at.word][] 
#              string word.count stop.at.word 
# 1:              one   1   0 
# 2:            two words   2   1 
# 3:           three words here   3   2 
# 4:         four useless words here   4   2 
# 5:      five useless meaningless words here   5   4 
# 6:     six useless meaningless words here just   6   3 
# 7:    seven useless meaningless words here just to   7   3 
# 8:  eigth useless meaningless words here just to fill   8   6 
# 9:  nine useless meaningless words here just to fill up   9   7 
#10: ten useless meaningless words here just to fill up space   10   5 
#           input output 
# 1:            one 
# 2:           two words 
# 3:         three words here 
# 4:        four useless words 
# 5:    five useless meaningless words here 
# 6:      six useless meaningless words 
# 7:     seven useless meaningless words 
# 8: eigth useless meaningless words here just  to 
# 9: nine useless meaningless words here just to fill 
#10:   ten useless meaningless words here just 

io non sono sicuro di aver capito la logica per output essere nulla per prima linea, ma la correzione banale, se davvero necessario, è lasciato a OP.