VTE isn't particularly fast or lightweight either way.
The developers also see it as being GNOME Terminal as a widget rather than a library. Leads to hard-wiring only what GNOME Terminal wants and not providing a way to do things another way. Many arbitrarily private APIs, etc.
I had a project providing a keyboard-centric terminal using VTE and we could never get them to expose the APIs we needed for keyboard-based text selection, etc. Handed it off to other people but they lost interest too, largely because it's so frustrating dealing with VTE.