Proton: A Few Steps Back

Proton, as a technology, has been over-delivering compared to our wildest expectations when it first launched. I have also been tracking with excitement every little piece of progress since last year, with my regular reports on new games working with it as well as the now well known graphs representing what looked like ever increasing progress to full compatibility.

However, here’s a new picture that shows that the recent state is suffering from a few setbacks:

To be fair, it’s not a huge change but it does look like it brings us back to where we were in April 2019 overall. When I analyzed in more details what was happening, I focused more specifically on the changes between June and August 2019, since they have a comparable unique number of games rated and a large difference in ratings.

Let’s first clarify whether or not the difference in ratings comes from new titles or existing titles.

I first checked the games which were common among these months.
There were 721 titles (a little more than 1/3 of games) rated both in June and August 2019. In June 2019, 42.6% of these games received a “platinum” rating in August 2019, it was only 40.6%. So there is a small decrease but that would not explain the gap we are seeing here.

I then checked for games that were not common to these months. (1344 in June, 1217 in August). Here the results are quite different. In June 2019, 56.4% of these games received a “platinum” rating, while in August 2019, it was only 46.4%. About 10% less. So this is where most of the gap comes from: newer titles.

There seemed to be a correlation with the release of 4.11 at the end of July 2019. For reference:

• Proton 4.11-1 released 31st of July 2019.
• Proton 4.11-2 released 9th of August 2019.
• Proton 4.11-3 released 27th of August 2019.

So the ratings in August are basically reflecting a mix of 4.11-1 and 4.11-2 Proton versions.

I reached Codeweavers with the above data and asked them for feedback. This is what they gathered:

Looking over this list reminded me there was a major regression in 4.11-2 that broke Unreal Engine 4 titles, which was fixed in 4.11-3.

Some of the popular games hit by this regression were, for example, Conan Exiles (which rating fell from Platinum to Borked), Mordhau (Platinum to Gold), ARK Survival Evolved (rated Borked during August 2019). All of these regressions were fixed in 4.11.3:

• Mordhau is now Platinum for all 4.11-3 reports,
• Ark: Survival Evolved gets Silver (BattleEye),
• Conan Exiles gets Platinum ratings again on 4.11-5 released very recently (our only reference since there were no 4.11-3 reports).

One of the casualties during that period was also No Man’s Sky, whose rating fell from Platinum to Silver. There is a good explanation for this one:

NMS also released a major update last month, so it may be that doesn’t work as well as it used to.

NMS is not the only one whose update caused major regressions. One of the latest, and more popular titles, Grand Theft Auto V, refused to launch after a recent update. There’s a long thread on Github that discusses how to fix it, but the key take-away is that vanilla Proton just cannot run it at all anymore. For now.

It also seems that for games we could compare between June and August, there’s an element of noise in the ratings. One good example is Overcooked:

• 3 ratings in June 2019: 2 Platinum ones and one Gold one, resulting in a median Platinum rating.
• 4 ratings in August 2019: 1 Platinum one, two Gold ones, one Borked (running on 3.7-8). This leads to a median Gold rating.

Has the game compatibility fundamentally changed in between 4.2-9 and 4.11-2 ? Hard to say. It could be, in this particular case, just noise due to individual differences or very specific configuration issues.

Codeweavers’ assessment of the situation is summed up by:

I think the lowered ratings here can be explained by the 4.11-2 regression, by better reporting from users, and by just bad luck of last month’s new titles not working.

As mentioned earlier, newer titles are certainly causing the ratings to drop, and that is outside of the control of Codeweavers/Valve as you can’t really fix issues you don’t know about yet. Now, if the assumption that there was no major regression apart from Unreal Engine in 4.11-2, then we should expect ratings in September to rise up a little, somewhere in between June and August 2019.

In some ways, I don’t think the idea of having regressions is very surprising. If you have ever used WINE extensively before the Proton era, you probably know that newer versions of WINE sometimes broke games that worked perfectly fine with previous versions. This is actually something I expected back when I interviewed James Ramey in late 2018: more regressions as Proton supports more and more games. This did not really happen, until recently: for the most part, in the span of a year, most games that worked well with Proton continued to work well and newer titles were supported.

It’s hard to draw any conclusions at this stage until we see how ratings evolve in the next few months, but if newer titles seem to be “less compatible” than before, this could result a slower pace of progress for the Proton project.

In any case, from our side, the best thing we can do is provide reports thru ProtonDB in larger numbers so that Proton developers get a good picture of what works and what breaks.

BoilingSteam lets you access our content for free, but writing articles is a constant investment. We don't use ads or sponsporship, help us make our activities sustainable by donating via Patreon or LiberaPay if you prefer it anonymous. You can follow what we do via our newsletter, our RSS feed, our Mastodon profile or our Twitter feed. We also have Peertube, Youtube and LBRY channels. If you'd like to chat, you can also find us on #boilingsteam:matrix.org. (what is Matrix?)