Gregory's ServerI'm just gonna ramble about codecs for a bit.
Codecs like H.264, AAC, PNG, JPEG, GIF, and even MP3, MPEG-2, and AC-3 are never going to go away. They're going to be with us forever.
Plenty of codecs have come and gone a la RealVideo (.rm), WMA, WMV, FPS1, H.263, DivX/Xvid/MPEG-4 Part 2, VP6, RealAudio (.ra), etc.
Then there are codecs that were pretty much dead on arrival like FLIF.
The image format wars are so fucking fucked right now. On the Chrome (and Firefox because they have no significant market share anymore so they just have to follow whatever Google decides) side you have WebP (AVIF is also starting to gain some traction) and on the Apple/Safari side you have JPEG2000. Safari has enough market share that CDNs will encode JPEG2000 images just for Safari.
The best solution is JPEG XL. However, adoption isn't looking so good.
https://caniuse.com/jpegxl
It looks like AVIF has won the image format wars.
https://caniuse.com/avif
Well, it certainly could have been worse.
I also just found out about Google's Lyra speech codec:
https://en.wikipedia.org/wiki/Lyra_(codec)
And Microsoft's new speech codec, Satin:
https://techcommunity.microsoft.com/t5/microsoft-teams-blog/satin-microsoft-s-latest-ai-powered-audio-codec-for-real-time/ba-p/2141382
Those are both really cool!
It makes me wonder what could be possible as we get more powerful NPUs and TPUs. Would be neat to use a neutral net to encode video. I guess it is already kind of possible. You could use a neutral net to examine a video and dynamically adjust the encoding parameters on the fly to get the best possible quality.
Actually, I wonder if that's what NVEC2 is doing. Once NVIDIA added tensor cores we got NVENC2. Encoding speed was almost halved compared to previous generations but the quality dramatically went up. At content delivery bitrates like 720p 30FPS at constrained VBR of 3.5 Mbps it'll deliver a higher quality video than x264 on the placebo preset. I bet that's what they're doing. Well, at least it's part of what they're doing. Would be nice to have an open source implementation tho.
Speaking of new possibilities for video encoding, I wonder if VVC/H.266 will learn their lesson from HEVC/H.265 licensing fiasco. Even if licensing is exactly the same as AVC/H.264 I wonder if the industry will adopt it the same as they did with AVC/H.264 or if they'll just embrace royalty free codecs like AV1 because they don't want the possibility of having to deal with all of that shit again with the next generation.
I mean, it seems the industry got burned pretty badly from the HEVC/H.265 licensing mess. I think they've kind of screwed themselves. The industry seems to be embracing royalty free codecs. You needn't look any further than Opus to see that. xHE-AAC outperforms Opus. Especially at very low bitrates. However, the industry sees Opus as good enough and it has received wide adoption while xHE-AAC has revived very limited adoption. And now for speech we apparently have neural net codecs that are royalty free and can work at absurdly low bitrates.
So, even though VVC/H.266 outperforms AV1 I could see the industry saying, "that's cool but this is good enough. We're not doing that again. We'll be making our next codec ourselves to."
I think our future is going to be filled with royalty free codecs that are standard and widely supported.
Thank you for reading my blog post.
internet.jpg
Codecs like H.264, AAC, PNG, JPEG, GIF, and even MP3, MPEG-2, and AC-3 are never going to go away. They're going to be with us forever.
Plenty of codecs have come and gone a la RealVideo (.rm), WMA, WMV, FPS1, H.263, DivX/Xvid/MPEG-4 Part 2, VP6, RealAudio (.ra), etc.
Then there are codecs that were pretty much dead on arrival like FLIF.
The image format wars are so fucking fucked right now. On the Chrome (and Firefox because they have no significant market share anymore so they just have to follow whatever Google decides) side you have WebP (AVIF is also starting to gain some traction) and on the Apple/Safari side you have JPEG2000. Safari has enough market share that CDNs will encode JPEG2000 images just for Safari.
The best solution is JPEG XL. However, adoption isn't looking so good.
https://caniuse.com/jpegxl
It looks like AVIF has won the image format wars.
https://caniuse.com/avif
Well, it certainly could have been worse.
I also just found out about Google's Lyra speech codec:
https://en.wikipedia.org/wiki/Lyra_(codec)
And Microsoft's new speech codec, Satin:
https://techcommunity.microsoft.com/t5/microsoft-teams-blog/satin-microsoft-s-latest-ai-powered-audio-codec-for-real-time/ba-p/2141382
Those are both really cool!
It makes me wonder what could be possible as we get more powerful NPUs and TPUs. Would be neat to use a neutral net to encode video. I guess it is already kind of possible. You could use a neutral net to examine a video and dynamically adjust the encoding parameters on the fly to get the best possible quality.
Actually, I wonder if that's what NVEC2 is doing. Once NVIDIA added tensor cores we got NVENC2. Encoding speed was almost halved compared to previous generations but the quality dramatically went up. At content delivery bitrates like 720p 30FPS at constrained VBR of 3.5 Mbps it'll deliver a higher quality video than x264 on the placebo preset. I bet that's what they're doing. Well, at least it's part of what they're doing. Would be nice to have an open source implementation tho.
Speaking of new possibilities for video encoding, I wonder if VVC/H.266 will learn their lesson from HEVC/H.265 licensing fiasco. Even if licensing is exactly the same as AVC/H.264 I wonder if the industry will adopt it the same as they did with AVC/H.264 or if they'll just embrace royalty free codecs like AV1 because they don't want the possibility of having to deal with all of that shit again with the next generation.
I mean, it seems the industry got burned pretty badly from the HEVC/H.265 licensing mess. I think they've kind of screwed themselves. The industry seems to be embracing royalty free codecs. You needn't look any further than Opus to see that. xHE-AAC outperforms Opus. Especially at very low bitrates. However, the industry sees Opus as good enough and it has received wide adoption while xHE-AAC has revived very limited adoption. And now for speech we apparently have neural net codecs that are royalty free and can work at absurdly low bitrates.
So, even though VVC/H.266 outperforms AV1 I could see the industry saying, "that's cool but this is good enough. We're not doing that again. We'll be making our next codec ourselves to."
I think our future is going to be filled with royalty free codecs that are standard and widely supported.
Thank you for reading my blog post.
internet.jpg