Post by elsandosgrande in Out of Touch! comments

Viewing post in Out of Touch! comments

Proton works as well and is probably the more practical approach. I don’t know about other distros, but it’s available as app-emulation/wine-proton in the main Gentoo repository at the very least.

Edit: Err, actually video playback doesn’t work (which I only noticed now); the last image shown/loaded just remains on screen until all videos have been skipped and the next bit of dialogue with its corresponding image is reached, at which point gameplay resumes normally. Looks like either switching to Proton-GE! or cloning Media Framework DLLs from a Windows installation might fix this, but I have yet to try either.

If the developers ever see this comment, it would be nice if the videos could be reencoded as VP8, VP9, or AV1: https://www.reddit.com/r/linux_gaming/comments/ve0kjw/a_developer_is_asking_what_codecs_are_supported/ https://forum.unity.com/threads/videoplayer-videos-do-not-play-on-steamdeck.1387941/

Edit two: So, when running OoT.exe from the terminal, the Wine standard output seems to indicate that this isn’t an issue with the Media Foundation, but with DirectX Media Object libraries being missing, specifically msdmo.dll. Winetricks does have a dsdmo verb, but it seems to only install the 32-bit version of the libraries: https://github.com/Winetricks/winetricks/issues/660

Edit three: So… After experimenting with a clean Wine prefix, it appears to me that at least some of the information regarding Wine and Media Foundation is outdated and that a lot more Windows libraries have been implemented in Wine since the default prefix was first initialised on my laptop, as both mfplat.dll and msdmo.dll are present now with Wine (Proton) 9 — there is output from mfplat itself in the terminal now and there are no errors regarding msdmo.dll — and the Intel VA-API driver is being called by GStreamer, which means not only is video playback working, but it’s working with hardware acceleration! Well, it would at least if it weren’t for the following GStreamer error right after initialising the VA-API driver:

0:00:01.483014858 834788 0x7a83fc0010b0 ERROR             audio-info audio-info.c:302:gst_audio_info_from_caps: no rate property given

</del>

Now, either the video’s audio is being screwy in some way or my GStreamer installation is being screwy, but at least the video codec can be ruled out as causing any issues, at least outside of Steam (and there sadly isn’t much information out there about Proton when used outside of the Steam client specifically).

(Hopefully) final edit: Read my response to Nikolay’s response. Tl;dr: Everything works just fine after switching to DXVK in spite of that GStreamer error (and one more error about a surface being detached or something).

Like Reply

Nikolay Bychinskiy1 year ago (6 edits) (+1)

We encode videos with ffmpeg using x264 codec with constrained baseline h264 profile and if it has audio it’s encodeded with qaac codec. If there are some issues with video playback for some reason you can try unpacking movies bundles and transcoding to VP8 and then bundling again with Unity 2019.4 for windows platform but there is no hardware acceleration for VP8 in Unity (even if platform supports hardware acceleration, the only exception is android where unity will sometimes have hw acceleration for vp8 videos).

Here’s ffprobe output for one of the videos, I assume rate refers to sample rate and ffprobe has no problem reading sample rate:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '.\teresa_trailer.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.3.100
  Duration: 00:01:44.17, start: 0.000000, bitrate: 7969 kb/s
  Stream #0:0[0x1](eng): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080, 7642 kb/s, 60 fps, 60 tbr, 60k tbn (default)
    Metadata:
      handler_name    : ?Mainconcept Video Media Handler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.3.100 libx264
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 317 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]

I can provide reencoded bundle but you will need to contact me on discord and I can’t guarantee how long it will take since I don’t have any scripts prepared for VP8 encoding and last time someone needed it was too long ago and we have some new videos + some old videos have been improved. We are not going to make VP8 default because it loses in every way - it works worse on low end hardware, it either is larger or has lower quality or both. It’s not worth for platform with barely any players. VP9 and AV1 are both not supported by Unity and also are significantly more computationally expensive so even if they were supported not worth it (and neither does it allow adding codecs to built in video player, entire video player has to be reimplemented entirely to add new codec and to make it reasonably performant it has to be native plugin so it’s a massive pain).

Like Reply

elsandosgrande1 year ago (5 edits)

Hi, and thanks for the info! I knew that AV1 was heavier on the CPU, but I wasn’t aware that VP9 was too, not to mention not being able to easily include videos in multiple codecs :/

So, this is probably gonna sound a bit silly, but video playback just started working flawlessly with full hardware acceleration once I switched from Wine’s built-in DirectX(/Direct3D?) 11 implementation to DXVK — it’s even using functionality called VideoEnhance in intel_gpu_top, which I’ve only ever seen one other application do and which I still don’t fully understand — so this ended up being a bit of a wild goose chase in the end. Between that and the only seemingly relevant information that I could find online being H.264-encoded videos not working in Steam games due to Steam likely not wanting to risk stepping on any patent owners’ toes, that is not enabling H.264 on Linux by default and instead opting for a mixture of reencoding videos on their servers and whitelisting specific games (and Media Foundation apparently not being supported/implemented in past versions of Wine), I apologise for the confusion 😅

By the way, I’ve only played through the first fifteen chapters thus far — yes, I skipped over a few video scenes without even realising that they were video scenes and thought that Wine or the game was acting up — but this has been the most beautiful and immersive visual novel I’ve played on itch thus far, so, to you and the rest of the team, thank you so much for creating this amazing game and I wish you all the best!

Edit: So, I just had a thought: Instead of the built-in DirectX 11 implementation being faulty, it could be that it just defaulted to my NVIDIA dGPU instead of my Intel iGPU, which would nicely explain why video playback returned nothing when it was decoding the video on the iGPU. Now, generally, OpenGL applications default to the iGPU on my laptop — yes, it did seem to fall back to OpenGL rendering according to the terminal output, and no, I didn’t check which GPU was in use — but, seeing as Vulkan applications manage to always default to the dGPU unless I force the iGPU, I could imagine Wine’s DirectX 11 implementation following suit in spite of using OpenGL in the end.

Before somebody asks: “Why didn’t you install DXVK right away‽”, well… I did, but MinGW64 being kinda fragile and me being kinda liberal with compiler flags on my installation of Gentoo led to any and all games crashing immediately upon trying to open any sort of window because of faulty stack smashing protection getting tripped. You can see the DXVK issue ticket here, but, in short, removing -fstack-protector-strong from my global CFLAGS (which I actually did a while ago but just didn’t rebuild MinGW64 in the meantime), adding -fno-stack-protector and removing -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 for MinGW64 specifically for good measure, and rebuilding DXVK fixed the issue for me.

Like Reply

Nikolay Bychinskiy1 year ago (4 edits) (+1)

Well, I was always saying that you need Wine+DXVK for game to work properly on Linux, no DXVK always lead to some issues with videos for people. Those who don’t have vulkan capable hardware need VP8 bundles but that’s very rare, pretty much everything in the past 10 years should support vulkan.

Speaking of patents - there is a way to avoid having to pay royalties for using h264 by using prebuilt OpenH264 binaries from Cisco. Limitations are that it only supports constrained baseline profile, must be downloaded on demand instead of bundled with software and must be optional (so there should be option for user to disable OpenH264). And I’m not 100% sure on that but I think if there is hardware codec then royalties are on hardware manufacturer rather than software that uses it. Video codecs are a patent minefield but all the royalty free codecs are designed by companies that only care about web and reducing traffic so they are all extremely computationally expensive, so h264 is still the best consumer codec that supports decent quality/compression while working well on low end hardware.

Like Reply

elsandosgrande1 year ago

I see. I didn’t know that OpenH264 couldn’t be bundled, although I guess in hindsight it makes sense so that Cisco can keep track of how much they need to pay in royalties.

By the way, is the Discord server the right place to report bugs/issues with the game? I have experienced the game crashing/closing unexpectedly when not in focus, but it’s very infrequent and seemingly completely random, so I haven’t encountered it on the few times that I’ve had the game running in the terminal as opposed to the itch.io client. If the server is the right place to go regarding this, I’ll join once I have at least some idea as to what’s going on.

Like Reply

Nikolay Bychinskiy1 year ago (4 edits) (+1)

Yes, currently you can only report bugs on discord server in #oot-bug-smashers channel. Keep in mind that work on current runtime is mostly stopped and all focus is on big engine upgrade that will come out next year so I’ll only fix if you can provide enough information and it’s relatively simple. There is a good chance this is a problem with Unity itself because the only thing that game does on focus change is muting sounds if that setting is enabled and also stop handling mouse events except for scroll (so that clicks that return focus to window would not be handled by the game).

Like Reply

elsandosgrande1 year ago

All right, thanks for the info and have a nice day!

Like Reply