I think this doesn't get talked about enough. If your input is a big run of data that is being checked/transformed in one shot, it works well. But, if you're likely to have to make a decision on several bytes of the input, SIMD will be the same or slower than the scalar method. It's not a magic "go fast" button.
That is not necessarily true. simdjson exists. But it's far from simple.
I have been told SIMD is good for data-parallel loops, like the GPU is, and that is true, but it can also be used piecemeal, unlike the GPU. Because it is just the CPU, you can read 32 unaligned bytes, scan for the index of the first space, and take a branch based on that.
I am speaking exactly of simdjson. If you look closely, you'll see that the high performance is really only achieved by skipping over large chunks of the input data. Which is great, if your use case is operating on a subset of data. But, if the application really needs to process every piece of the entire input, there's no performance win.
Think about it, JSON structure is almost entirely made up of single bytes ({ } [ ] , : "). If you're not skipping over stuff, you're not going to get away from having to examine those bytes and branch on them. For something like `{"a":1,"b":2}`, the SIMD scan has a bunch of overhead to find the interesting parts and then you go back and practically have to reprocess every single byte.
I think the big miss is that the former is achievable far more often than people imagine, which leads to settling for the later, aka pessimization. Reducing your allocations from millions per run to handfuls per run during initialization is effectively a "go fast" button, and is generally more reliable. It is very common for teams to spend big effort getting a 2-3x speedup on allocation-heavy code by disabling branches when a 100-1000x speedup can be had if restructuring allocations is a strategy under consideration (even before the extra 4-8x you might see if you go all the way to hand-tuned SIMD).
I think this doesn't get talked about enough. If your input is a big run of data that is being checked/transformed in one shot, it works well. But, if you're likely to have to make a decision on several bytes of the input, SIMD will be the same or slower than the scalar method. It's not a magic "go fast" button.