Immediately following the localization release last year, we started development of the mobile ports of Banner Saga. I started on iOS first because much of the work done there would apply directly to Android.
One of the first issues to be resolved is the problem with loading all the GUI elements into the game. Up to this point, we had loaded GUI elements as Flash SWFs at runtime, as they are needed. However, a bug with the way the SWF Loader works on iOS prevents this from working on iOS. Basically, the SWF Loader throws an error if the loaded SWF contains any ActionScript Bytecode (ABC). Even though there is zero actual functional ABC included in our GUI SWFs, the ‘exported symbol’ functionality creates boilerplate empty class definitions, which are technically ABC. Instead of being sensible and recognizing the difference between actual functional ABC and empty boilerplate definitions, the Adobe Loader just craps out.
The solution we came up with is to link all the GUIs with the game executable at compile time. This has the potential to increase the size of the executable by 100 MB or more, but we have implemented a system that allows us to strip out the large bitmaps from the GUI SWF, and load them dynamically at runtime. The work of changing over the GUIs in this way is something that had to be done one by one, with a good bit of manual labor and testing at each step. It's worth noting that Adobe provides a mechanism to accomplish this dynamic loading of SWFs, but I opted not to create a dependency upon this. My reasoning was to avoid a dependency on an infrequently used and esoteric feature of Adobe AIR over which I have no control. If you want to read more about it, you can click here: http://blogs.adobe.com/airodynamics/...r-apps-on-ios/
Another issue that arose is the limitation of iOS application size. The iOS application file, called IPA, has a maximum size imposed by Apple of 2 GB. Our game with all its assets was around 2.5 GB in size. Some of the biggest assets are the videos and sound, followed by the numerous massive bitmaps used in all the scenes. We deliberately made our textures large to support retina displays and future large displays. The GUIs and scenes are all authored for a native resolution of 2731x1536 (that would be the resolution of a Retina iPad if it were extended horizontally into Cinemascope aspect ratio). We reduced the resolution of the cinematic videos, which saved us quite a bit, and increase the compression of the sound and music. Aside from this, we went through all the assets to find any stuff that really doesn't need to be present. Additionally, we made changes to the way the animation clips were stored and compressed.
Another issue we have hit is that our video playback doesn’t work on iPad. We are currently using flash.media.Video, and a different technology called StageVideo is available on mobile, and supposed to be very performant. However, like many things Adobe AIR, this feature did not work as advertised. After struggling with it for some time, I found a way to continue using flash.media.Video by changing the video encoding from mp4 to flv. This required some content pipeline changes to make sure we package the mobile builds with different video assets than the desktop builds.
The biggest issue we faced, though, was performance. Up to this point, the game had been rendered using the flash display list. The flash display list renderer is totally CPU bound, which works fine on modern PCs, but on mobile, the performance is not even in the ballpark of acceptable. To move the rendering work off of the CPU and onto the GPU, I refactored the entire rendering system to use a technology called Starling, which is built on top of Stage3D. This means that all rendering of the game worlds now goes through the GPU, resulting in an absolutely massive performance increase. A really nice side effect of this refactor, is that the same fast rendering path works on PC and Mac. When we updated the Banner Saga on Steam earlier this year, you got the benefit of these changes, resulting in a much faster game framerate.
A downside to this approach is a hard limit to the allowable size of bitmap textures. We had been using quite large textures on PC and Mac, but on mobile with Starling we had to set the limit to 2048x2048. Some devices support higher than this, but this is a good 'least common denominator' to work toward. To support this, I had to write some tools to go through all the scenes and automatically slice up textures that are too large.
On a side note, Adobe Scout CC is a fantastic tool for analyzing frame-by-frame performance issues in Adobe AIR apps, on desktop as well as mobile. You can get more information about it here: https://creative.adobe.com/products/scout
The final challenge to mobile porting was memory management. A modern iOS device has 1 GB of memory available, so to be safe you need to stay quite a bit below that, say 700 MB. For 32 bit Windows, we had to ensure that the game never exceeded about 1.2 GB of active memory, so we were in the ballpark.
The first pass was to make sure everything was getting cleaned up properly between scenes, and ensuring that we weren't loading any unnecessary data. The biggest consumer of memory, by far, turns out to be the combat animations. Especially with large units like Bellower on the board, the memory usage could quickly get out of hand. In fact, the final Bellower fight ended up being our primary benchmark to measure memory pressure.
The biggest issue is that for each character, we pack all frames of animation onto a few large textures (sprite sheets). This means that loading the animations is pretty quick because the number of files is few, but it also means that we keep all that texture data in memory whenever the character is loaded. These sprite sheets were stored as individual PNG files. To resolve the issue, we created a new file format for our animation data. We individually zlib-compress each frame of animation, then pack them all sequentially into a buffer of raw bytes. We store the buffer offsets and sizes of each frame with the animation information. Then, at runtime, we can keep all the frames compressed until needed. This has the negative side effect of incurring the CPU expense of a decompression whenever the character animation frame changes. On iOS and PC this is not a big issue, although Android devices do suffer much more. We can still disable the compression altogether if a device like a PC has plenty of memory, or even in the future take a hybrid approach where we predictively uncompress groups of animation frames within some fixed allowable budget of memory.
Jason Maltzen of HiddenAchievement helped us out quite a bit getting the mobile ports completed. He wrote all of the ANEs for integration with the platforms (Game Center, iCloud, Google Play, Game Circle, etc…) as well as ported the FMOD ANEs to those platforms. He has taken on much of the work involved with porting to Android and the ongoing support thereon.
Next: Technical Blog #9 (Gamepad Support)
Previous: Technical Blog #7 (Localization)