François CholletVerified account

is there also an easy way to do small variations? e.g., run to the two sides of each word, without the word?

not entirely sure what you mean. Please give an example?

z_i = concat(LSTM(x_1,..,x_{i-1}), LSTM(x_n,...x_{i+1}))

you're talking about variable length sequences?

variable length I take for granted. I meant representing the context of the word without the word itself (or sim tweaks)

also there is some structure in the joint distribution of the left and right contexts that should be modeled

yes, presumably a multi-layer perceptron on top of the concat should capture this to some extent.

if you want to use Keras, easy way is to do the complicated splitting in Numpy & use a multi-input model.