MoSal

/tmp is still tmpfs here, but I prefer this (below) nowadays for usages that are not typical /tmp usages:

  n=$(cat /sys/class/zram-control/hot_add)
  echo zstd > /sys/block/zram${n}/comp_algorithm # change that if you like
  echo 8G > /sys/block/zram${n}/disksize # larger disk probably needed for /target
  mkfs.ext4 -O '^has_journal' -L TMPZ -m 0.001 /dev/zram${n}
  # discard is important for auto-trimming
  mount -o noatime,discard /dev/zram${n} /tmpz
  chmod 777 /tmpz
  chmod +t /tmpz
  rm -rf '/tmpz/lost+found'

This is a part of my zram devices initialization script, a bunch used for swap as usual, then ending with this. modprobe zram num_devices=0 works if one likes to hot_add all devices.

From zram you get a block device with builtin compression. From ext4, you still get features like creation time, truncate and fallocate support, ...etc. And with discard, ram usage will be limited to used space (like tmpfs). Also ext4 is used without a journal to avoid what would be useless overhead in this use-case.

I've been working on something similar-ish on and off.

There are three (good) solutions involving open-source models that I came across:

• KenLM/STT
• DeepSpeech
• Vosk

Vosk has the best models. But they are large. You can't use the gigaspeech model for example (which is useful even with non-US english) to live-generate subs on many devices, because of the memory requirements. So my guess would be, whatever VLC will provide will probably suck to an extent, because it will have to be fast/lightweight enough.

What also sets vosk-api apart is that you can ask it to provide multiple alternatives (10 is usually used).

One core idea in my tool is to combine all alternatives into one text. So suppose the model predicts text to be either "... still he ..." or "... silly ...". My tool can give you "... (still he|silly) ..." instead of 50/50 chancing it.

This will not work on Unicode

Correct.

Greek and Arabic have cases as well

Who told you that?

Thank you for working on this. I found Okhsl to be of great utility.

Okay. I updated mold to v2.0.0. Added "-Z", "time-passes" to get link times, ran cargo with --timings to get CPU utilization graphs. Tested on two projects of mine (the one from yesterday is "X").

Link times are picked as the best from 3-4 runs, changing only white space on main.rs.

lto="fat"	lld	mold
project X (cu=1)	105.923	106.380
Project X (cu=8)	103.512	103.513
Project S (cu=1)	94.290	94.969
Project S (cu=8)	100.118	100.449

Observations (lto="fat"): As expected, not a lot of utilization of multi-core. Using codegen-units larger than 1 may even cause a regression in link time. Choice of linker between lld and mold appears to be of no significance.

lto="thin"	lld	mold
project X (cu=1)	46.596	47.118
Project X (cu=8)	34.167	33.839
Project X (cu=16)	36.296	36.621
Project S (cu=1)	41.817	41.404
Project S (cu=8)	32.062	32.162
Project S (cu=16)	35.780	36.074

Observations (lto="thin"): Here, we see parallel LLVM_lto_optimize runs kicking in. Testing with codegen-units=16 was also done. In that case, the number of parallel LLVM_lto_optimize runs was so big, the synchronization overhead caused a regression running that test on a humble workstation powered by an Intel i7-7700K processor (4 physical, 8 logical cores only). The results will probably look different running this test case (cu=16) in a more powerful setup. But still, the choice of linker between lld and mold appears to be of no significance.

lto=false	lld	mold
project X (cu=1)	29.160	29.231
Project X (cu=8)	8.130	8.293
Project X (cu=16)	7.076	6.953
Project S (cu=1)	11.996	12.069
Project S (cu=8)	4.418	4.462
Project S (cu=16)	4.357	4.455

Observations (lto=false): Here, codegen-units becomes the dominant factor with no heavy LLVM_lto_optimize runs involved. Going above codegen-units=8 does not hurt link time. Still, the choice of linker between lld and mold appears to be of no significance.

Observations (lto="off"): Same observations as lto=false. Still, the choice of linker between lld and mold appears to be of no significance.

Debug builds link in <.4 seconds.

codegen-units=1, debug=true, varying lto

lto = "fat"

lto = "thin"

lto = false

lto = "off"

codegen-units=8, debug=true, varying lto

lto = "fat"

lto = "thin"

lto = false

lto = "off"

mold appears to be similar but not faster than lld.

With the caveat that this is not a proper benchmark since:

• I didn't measure link time alone.
• I didn't bother running each case multiple times picking the fastest run (since I perceived the differences to be insignificant).

And a side note, lto = false appears to be practically useless.