Daniël de Kok    

@danieldekok

Researcher in computational linguistics, c(rust)acean 🦀, Nix/NixOS ❄️, occasional tinkerer with electronics. Dad of a Lego queen 👸. Opinions are my own.

Joined March 2008

Tweets

You blocked @danieldekok

Are you sure you want to view these Tweets? Viewing Tweets won't unblock @danieldekok

  1. Just pushed the version 0.1.0 of the `sentencepiece` crate, which provides a rustic interface to the sentencepiece unsupervised text tokenizer.

    Undo
  2. Back home from a great , many thanks to the organizers!

    Undo
  3. Nix is so extremely nice. The PyTorch 1.4.0 headers are not compatible with gcc 9. Switch to the gcc8Stdenv build environment, and we can pretend as if gcc 9 never happened.

    Undo
  4. Jan 25

    direnv 2.21.0 has been released. This is a massive release! * `.envrc` files are now loaded with `set -euo pipefail` which is probably going to expose issues in existing scripts. * Ctrl-C now actually works during reload in bash and zsh. And more!

    Undo
  5. Jan 25

    Our and bindings for are now compatible with PyTorch 1.4, updated opam package/crate are available! A bunch of examples can be found on GitHub, RNNs, GAN, Reinforcement Learning...

    Undo
  6. Dear lazyweb, I mean fellow crustaceans. I am working on a crate that binds a C library, but can only reasonably unit-test it by providing a ~500KB data file. Would you find this acceptable for a tiny binding? Alternative: feature-gate those tests.

    Undo
  7. Australia: you have just experienced the future.

    Show this thread
    Undo
  8. For some reason, I haven't been invited to Davos this year. Was it something I said? 🤔

    Undo
  9. 0.45 dependency parsing LAS improvement on Dutch, with only half the parameters. That's what I call a productive week 🥂. Enjoy the weekend!

    Undo
  10. Jan 15

    Firefox market share vs Mozilla Foundation chair salary (2.5 million/year in 2018)

    Show this thread
    Undo
  11. Jan 9

    1/7 Did you ever need to run a piece of C# code on Windows 3.11? Me neither, but I did it anyway. A thread.

    Show this thread
    Undo
  12. Finally made the switch from Tensorflow to PyTorch. The tch-rs bindings for libtorch are really nice. Learning the libtorch API and porting the BERT model from Hugging Face Transformers to Rust was really quick:

    Undo
  13. I am very happy that we use far less gas for heating/hot water than the average of all house types. But that bar chart is weird. The deltas are probably proportional, but the bars themselves…

    Undo
  14. The final, finetuned models, use each layer almost equally. The interesting patterns here seem to be: for lemmatization, almost all layers are used equally. For the other tasks, use increases per layer. 4/4

    Show this thread
    Undo
  15. However, for morphological and POS tagging, it seems that for ML BERT the classifiers rely most on the initial/middle layers, whereas for BERTje more uniformly on all the layers. 3/4

    Show this thread
    Undo
  16. This can give hints where information is represented in the initial encoder. Interestingly, the information seems to be distributed quite differently between ML BERT and BERTje. In both, the last layers are used most for dependencies and the initial layers for lemmatization. 2/4

    Show this thread
    Undo
  17. More Dutch BERT explorations: since we used scalar weighting, we can see what layers are used per task. As is common in finetuning of pretrained models, we freeze the encoder weight for the first epoch, to avoid that large softmax gradients 'destroy' the encoder 1/4.

    Show this thread
    Undo
  18. And then your 5yo daughter walks into the room at 4 am, shouting “good morning everyone”. Turns out she thought her clock said 9am. :oops:

    Undo
  19. I played Commanche in the early 90ies, but never realized the the Voxel Space rendering algorithm is so simple. Cool!

    Undo
  20. 23 Dec 2019

    Hot off the press: Bertje. We collected a large and diverse corpus of Dutch and trained a monolingual BERT model. The model is available for download. Paper: joint work by me Gertjan van Noord & Malvina Nissim

    Show this thread
    Undo

Loading seems to be taking a while.

Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.

    You may also like

    ·