If you read my previous mutterings about AMD’s Mantle API and Star Swarm, you might remember that I mentioned not being able to test it out in Battlefield 4 as I didn’t own the game. That wasn’t intended to be a plea for freebies, but AMD read the article and said “Hey, would you like to see how Mantle works in Battlefield 4? Here’s an Origin code.”
As it happens, yes I do. So thanks to AMD for that code. I’ve now been fully paid off and will be writing non-judgmental things about Mantle. Wait, no, the other thing! I’m going to remain objective and do some more benchmarking.
I don’t know, Pac, 50 fps is okay.
The last piece provided a bit of an overview of what Mantle is, what API (application programming interface) means and why this stuff is of interest to PC gaming, so I’d recommend reading at least the first few paragraphs of that one if you want a basic overview. Here’s the super short summary: Mantle is AMD’s alternative to DirectX/OpenGL, tailored specifically for the HD 7000/8000 and R7/R9 range of graphics cards.
Drivers for this stuff are still in beta (Catalyst 14.1 is what you’ll need if you want to try the system out for yourselves) and the blurb for that release states that further optimisation in Battlefield 4 is expected for the majority of Mantle-ready cards. Specifically, to only expect “limited gains.” When you factor in the now notorious instability of Battlefield 4 to begin with, you’ve got a whole lot of caveats.
It also helps if you completely uninstall previous AMD drivers before sticking 14.1 on your system. At first I was lazy and just dumped them over 13.11 like a big driver slob, then paid the price as nothing really worked properly. It’s unofficial, but I’d recommend using Display Driver Uninstaller to give you a fresh base for new drivers. If you’re unsure about this stuff make back-ups before diving in, but it’s always been pretty straightforward for me. It clears up registry left-overs rather neatly.
This one isn’t from my testing, it’s just a regular ol’ Battlefield shot.
So, once again, here’s my PC set-up and the one used to generate all the findings and numbers that will follow:
- i3-2100 CPU at 3.10 GHz
- Asus 2GB HD 7870
- 8 GB of RAM
It’s no beast, but capable enough to do the job in most games. Might also make a change to get some numbers from a more moderate card than the R9 290X’s the dedicated tech sites all seem to be using, eh? For Battlefield 4, I had all video settings on the default “High” with no added MSAA (because that seemed to put me in the 50-70fps range enough of the time) and the resolution at 1980×1080.
The Battlefield 4 console has some handy commands you can use to help compare DirectX and Mantle APIs, so here’s what I used:
Render.DrawscreenInfo 1 (This just confirms whether you’re using DX or Mantle and a few other system specs.)
PerfOverlay.DrawFPS 1 (Sticks an FPS counter in the top right corner.)
PerfOverlay.DrawGraph 1 (Gives you a nice graph monitoring CPU and GPU usage in the bottom left corner, you can see it in my screengrabs.)
PerfOverlay.FrameFileLogEnable 1 (Starts a .cvs log of your frame times which can be found in User/Documents/Battlefield 4, stopped by using the same command but with a 0 rather than 1 at the end.)
Once I’d generated some .cvs files, I dropped them into the Google Docs spreadsheet tool and took a frame time average for the benchmarking period. Converting that into FPS (1000/frame time average) gave me an average FPS.
I’d hoped to record some video too, but at present the only reliable method of capturing Mantle that I could find was using Open Broadcaster Software and running Battlefield 4 in borderless windowed mode. Unfortunately that tanked my frame-rate, even when encoding at a rubbish bitrate. So you’re stuck with text and numbers I’m afraid.
Here I am, looking at some trees. Exciting!
Since Battlefield 4 has a map quite literally called “Test Range,” that seemed a reasonable place to start some tests. It’s not the most demanding map by any means, but I thought it would provide a nice baseline of data.
Taking the exact same actions for both the DirectX and Mantle benchmarks would’ve been impossible, but I tried to do pretty much the same thing each time. I deployed at point A, ran around the central building to point C, tracked back to point B, got in a helicopter and (in true Battlefield fashion) crashed it into the center of the island.
Here’s what the numbers had to say about that:
TEST RANGE
DX 11: Average Frametime = 11.33, FPS = 88.3
Mantle: Average Frametime = 11.64, FPS = 85.9
Mantle, in fact, came out slightly worse here. That may be due to slight variations in my actions. It could be the result of beta drivers, or my HD 7870 not yet being properly optimised for the game, or just Battlefield 4 behaving inconsistently. Or it could be that Mantle just wasn’t doing much for me here. The 2.4 FPS variance is pretty negligible, but after the relative success of the Star Swarm stress tests I was surprised to see Mantle coming off worse.
DirectX didn’t enjoy the helicopter showing up.
Next, I ran through a certain section of the Shanghai mission from the game’s single player campaign. Effectively, I started benchmarking at the point just after leaving the car at the start of the mission, when Pac is unlocking the gate. Then I ran down that alley, climbed the ladder, looked across the square at the hotel and the chopper, climbed down the next ladder and ran across the square to the hotel doors.
Since the rooftop overlooking the square is a somewhat demanding scene (at least on my system) and looking at walls while climbing ladders is about as undemanding as it gets, I thought this would be a relatively varied test of performance.
Here’s what happened:
SHANGHAI HOTEL DASH
DX 11: Average Frametime = 16.58, FPS = 60.3
Mantle: Average Frametime = 15.33, FPS = 65.2
This time Mantle came out in front, and with a similar sized margin to the one Star Swarm seemed to provide. However, that doesn’t quite tell the whole tale. While using Mantle, I was getting regular, huge spikes in CPU usage (shown by the in-game overlay graph,) and when these occurred my frame-rate would nose-dive. DirectX had a couple of CPU spikes too, but at what I imagine are designated checkpoint/loading type zones.
The real world upshot was that though Mantle felt smoother than DirectX when it was working, it was permeated by these annoying frame-rate drops and stutters at semi-regular intervals. Again, this could either be an issue or conflict with my system, beta drivers lacking in optimisation or Battlefield 4 being its usual weird self. I’m not really capable of speculating beyond reporting what I saw.
Other users with HD 7000 series cards have reported decent gains from using Mantle, but some have run into similar CPU spiking issues.
A nice flat graph, but you can see those CPU spikes. A few of them went off the chart.
Multiplayer matches would have too many variables for anything approaching a consistent benchmark (not that the ones I did were an exact science either,) but I jumped into a couple of games on the demanding Lancang Dam and Paracel Storm maps just out of interest.
Casual observation seemed to support what I’d already charted in single player. When using Mantle there seemed to be a small frame-rate boost (the “limited gains,” AMD themselves stipulated) and GPU usage was much more consistent, but CPU spikes kept interrupting play and sending my FPS spiraling for a second or two. So while DirectX seemed to be working my GPU harder (if the in-game graph was any indication) for equivalent or worse frame-rates, it was much more tolerable because there were fewer dramatic stutters.
As I’ve noted throughout the article, it’s almost impossible to tell whether Mantle’s performance in Battlefield 4 is down to beta driver issues, lack of optimisation, the game itself, or something involving my own system.
At this early stage, I’m not too disheartened. If Mantle is still struggling for consistent returns after several driver updates then that would be cause for concern. For now, it’s an interesting option for AMD users to experiment with and something with considerable potential for the future. The Mantle-ready Thief will be the next big test, so we’ll see whether AMD has improved optimisation across their compatible GPU range by the time that game comes out at the end of the month.
