PDA

View Full Version : Technical Blog #0



John
05-11-2012, 04:15 PM
http://stoicstudio.com/images/blog/john_and_victor200x150.jpg (http://stoicstudio.com/forum/album.php?albumid=2&attachmentid=13)The past week has been an exciting time! We hired our first programmer, which is an important milestone. Victor Ude started on Monday, and is sitting beside me as I write this, hard at work.

Victor is currently working on our FMOD integration. FMOD is a 3rd party sound and music engine for games. It provides a great deal of power for scripting and manipulating sounds in game. FMOD comes with a fantastic tool for sound engineers. FMOD will be used by Kpow and Austin Wintory to implement the sound and music in the game moving forward.

http://stoicstudio.com/images/blog/FMOD_Designer_512x288.jpg (http://stoicstudio.com/forum/album.php?albumid=2&attachmentid=10)

Victor is sorting the intricacies of writing an Adobe Air Native Extension (ANE) for allowing a C/C++ library like FMOD to interoperate with as3 libraries. As with any cross-language bindings, ANE, a relatively new technology, has little quirks and annoyances that I am very happy Victor is dealing with.

I’ve been working on a variety of tasks. For the past few days I’ve been getting GUI screen roughs mocked up in the game. Arnie has been painting the art assets for it, and I am getting the basic functionality hooked up. We have roughly 12 full screen GUIs to do, and I am going through them one by one and getting them working. This is all to pave the way for our GUI Designer, who will start working with us remotely from San Francisco on Monday.

GUI integration has been an enjoyable challenge. The GUIs are painted by Arnie in Photoshop, then the functionality is setup and programmed in Flash, and I have been working on some as3 frameworks to be reused across all the GUI screens to make everything consistent and easy to integrate into the game. I hope to have this GUI architecture fairly firm by the time the GUI Designer starts. Having this system architected in advance will help prevent us from going down ratholes that require substantial reworking/refactoring in the future.

Upgrading our website and forum hosting took some time last week. When we first rolled out our forums, we were using a shared hosting provider that had been up to that point just fine for our website and wordpress blog. However, as all of you know, the deluge of forum users crushed our shared hosting pretty quickly. VBulletin’s database resource usage far exceeded the available resources on the shared hosting. Almost all of our issues were as simple as hitting the database connection limit. Fortunately, our hosting provider allowed us to migrate our website and database into Virtual Private Servers, which allow us to scale up and use as much resources as we need. I now have some nice little slider bars where I can allocate more memory whenever I like.

We have a couple of remote volunteers who are helping to set up our Continuous Integration/Build Server. We are going to be running a Jenkins/Hudson service in the Amazon Web Services EC2 cloud. Since our game is cross-platform, we need to run one MacOS server and one Windows server, so we can produce the executable packages for each platform. That work is going slowly because both of the volunteers have day jobs, but it should be up and running prior to starting Alpha.

I have been taking the first steps to setting up our Game Server and Game DB. I am, again, using AWS EC2 to run an Ubuntu linux server, on which I will run our Game Server. The Game Server itself is a Java application using the Glassfish Jersey implementation of the Java API for RESTful web services. I am building out our DB using Amazon SimpleDB, which is a very nice key-value non-relational database. As you might tell from its name, it is pretty simple to use. Despite being non-relational and non-schema, it does support SQL query syntax.

We have a second programmer starting on Monday. His name is Jeff Uriarte, and he will be working on some tools for us. First up is the tool for creating the parallax scenes that are used in combat, travel, and city environments. We have a lot of these to make, and the tools will help streamline the process. For some reason the other guys don’t like the idea of building them by hand as 20 page JSON files :)

Next: Technical Blog #1 (http://stoicstudio.com/forum/showthread.php?121-Technical-Blog-1)

JokerAR
05-11-2012, 04:59 PM
If you can pledge to making one or two of these updates a month I can pledge to trying to understand more than 40 percent of the next one.

Thanks for the updates- as ever it's nice to see such hands on interactions with the fans!

edit: the above is in no way a suggestion to dumb down the updates- the warning for those of us without any programming backgrounds at all is clear in the title!

stelly
05-11-2012, 05:14 PM
Nice post John :D

Are you sitting higher because you are the leader of the clan and you can look down on your minions?? ;)

Stelly

Michael_Kpow
05-11-2012, 05:31 PM
Aha! Great to see FMOD progress!
Also hello Victor! Sounds like we may be communicating frequently in the future.
Exciting update! Geeky, but exciting :p

LoliSauce
05-11-2012, 06:17 PM
Nice post John :D

Are you sitting higher because you are the leader of the clan and you can look down on your minions?? ;)

Stelly

Speaking of minions... (http://www.youtube.com/watch?v=Ocm8QdNR_d8#t=2m03s)

Anyway, this is hype. I'm excited to hear how things progress.

belamoor
05-11-2012, 06:58 PM
Wow, that's an incredibly detailed update, wish i could fully understand it. :) Like why are there 12 different versions of GUI for a game, is that a resolution thing ?
Overall, nice to see plans setting in motion. Stoic is definitely growing.

lamaz
05-11-2012, 08:48 PM
Yay a technical blog! And I'd imagine the GUI amount comes from a few different GUIs(battle, travel etc.), which will be made available for all of the different platforms. Tablets GUIs for example require some tweaking to make it work.

On the subject of the GUI. What kind of are you going for? Are you going to lay it all for the player to see or are you going for more minimalistic solution of hiding some information behind some easily accessible sub-menus? For example will the abilities of characters in combat be in one menu that is always visible or will they pop-up when you select the character you want to use?

I bet it's quite challenging working with all those different platforms with such a small team. It's sometimes really hard to just wrap your brains around of simply using Linux, Windows and MacOS at the same time, let alone combined with all the different programs and programming languages. Good luck!

John
05-11-2012, 11:19 PM
1. I'm not sitting higher, I'm just 11 feet tall.

2. Yes, the number of GUIs represents the different screens -- assembling your party, upgrading your characters, hiring new characters, etc...

3. The GUI is intended to be as clean as possible but with very few sub-menus. How's that for having it both ways?

4. Yes, the multiple platforms do present a bit of a challenge. You have to configure everything to properly build and run, multiple times and places, and try to keep from breaking one accidentally while you work on the other.

John
05-11-2012, 11:25 PM
5. I would like to do a blog update weekly, but we'll see how the cadence rolls out as we go through production.

LoneGunman
05-12-2012, 03:53 AM
Nice update. And, as a tech guy, I can (thankfully) understand 100% of it. :) I'd be curious to see how things go with the AWS/simple DB set up. Been debating how to set up the prototype I'm working on. I really want to use a per-cycle cloud system like AWS or Azure (I'm mostly a .Net guy but I'm flexible :) ). I'm actually hoping to do more of a burst model and spin up resources as needed via a manager process that just monitors different slices running on AWS and allocates new slices dynamically when something (CPU/memory/network) hits a threshold.

Chaille
05-12-2012, 08:46 AM
Turboencabulators.

I like the photo. The ceiling of the goat shack slopes down on John's end of the room, making him even more tall. There appears to be a fan attached to the wall by his head, blowing directly into his right ear. And Victor knows better than to look him in the eye.

stoicmom
05-12-2012, 08:51 AM
5. I would like to do a blog update weekly, but we'll see how the cadence rolls out as we go through production.

Go John!!

lamaz
05-12-2012, 09:07 AM
Oh wow once a week would be nice. Are you going to make all of the updates or will you take turns?

smj123
05-12-2012, 10:09 AM
Exciting to see examples of the realization of the KS money being used. Keep the info flowing.

Mudfly
05-12-2012, 11:07 AM
Thanks for the update, an please keep 'em coming!

hatim
05-12-2012, 11:13 AM
Great post! Looking forward to reading every single one you write. Thank you so much for taking the time to do so.

Which source control tool do you use? do you use it for both art/music assets and source code? How do you keep track of tasks/bugs across teams?

It is great to see that a continuous integration environment is in the works. Are you doing any kind of scrum based development?

John
05-12-2012, 11:29 AM
We are using mercurial, hosted on bitbucket. Personally I think git/github is the best available. However, I feel that mercurial hides its complexity a little better than git, so it may be better for non-programmers to use. In practice they are so similar it doesn't matter that much. github is a better site than bitbucket -- BUT... github has very limited repository sizes, even for paid accounts. So github is not really suitable for storing art assets. Which should answer your other question -- yes we are using source control (mercurial/bitbucket) to store art/music assets as well as source code. Different repositories, but all in the same system.

In terms of bug tracking, we haven't gotten to that yet. Anything that needs to get fixed we just mention to each other, or create a task as described below. Once we get going with more testers and moving parts, we'll probably use something hosted like JIRA, or even use bitbucket's built in issue tracking.

We aren't really doing scrum per se, but I would certainly classify what we are doing as agile. I have a whiteboard with sticky notes in 2 columns: Backlog and In Progress. We arrange them in descending order of priority, and shuffle them around as often as necessary. I also have a spreadsheet with all the tasks broken down with work estimates so we can take a wild guess when we might be finished.

JW


Great post! Looking forward to reading every single one you write. Thank you so much for taking the time to do so.

Which source control tool do you use? do you use it for both art/music assets and source code? How do you keep track of tasks/bugs across teams?

It is great to see that a continuous integration environment is in the works. Are you doing any kind of scrum based development?

stelly
05-12-2012, 11:39 AM
1. I'm not sitting higher, I'm just 11 feet tall.



pffft... Midget :D

Stelly

Victor
05-12-2012, 06:16 PM
hello Victor!

Hello! ^_^

Eberict
05-13-2012, 07:06 AM
I don't understand this, but I will try very hard to, and I love that you posted it! Going to run it through a friend of mine who understand the programming aspects better. Your high-thane seat is impressive, but doesn't it feel strange being so close to the ceiling?

And hello Victor! I like your Makai Kingdom avatar. You need a better title than "Junior Member" though!

Troll
05-13-2012, 07:42 AM
Very glad to hear of the advancement of the develeoppement. I hope you guys have as much fun developping TBS as we will have playing it.


1. I'm not sitting higher, I'm just 11 feet tall.

I take it you're the model for hte Varls then.

Avantre
05-13-2012, 08:44 AM
Once we get going with more testers and moving parts, we'll probably use something hosted like JIRA, or even use bitbucket's built in issue tracking.

We use JIRA in the application (not game) development team I'm a part of. It does the job well enough.

Victor
05-13-2012, 12:23 PM
And hello Victor! I like your Makai Kingdom avatar. You need a better title than "Junior Member" though!

Hello, thanks, and I'm working on it :p

Suzie Q Sailaway
05-13-2012, 10:32 PM
! Thank you for the interesting und informative post! Makes me even more excited for the game (if that's actually possible). :)

AugustSnow
05-14-2012, 02:03 AM
Great update! makes me glad i'm a backer. Here's another vote for another one whenever you can post it...

Kyrosiris
05-14-2012, 05:56 AM
I will agree with the crowds that more Technical Blogs would be great, but I'm of the opinion that I'd rather have them when there's something constructive to put into 'em, instead of on a set schedule. Having TB#2 be "Well, we wrote code. It compiled. Cool." just seems like it's taking your time away from, well, writing code (and having it compile). :P

Thank you very much for this one, however!

Troll
05-14-2012, 06:42 AM
I will agree with the crowds that more Technical Blogs would be great, but I'm of the opinion that I'd rather have them when there's something constructive to put into 'em, instead of on a set schedule. Having TB#2 be "Well, we wrote code. It compiled. Cool." just seems like it's taking your time away from, well, writing code (and having it compile). :P

Thank you very much for this one, however!

This is exactly what Alex from the Starfarer devs wrote on the blog when people asked for more frequent updates. And that is totally true, give us news when there are, and not the way the TV does when nothing happens.

Gyro
05-14-2012, 09:44 AM
I really enjoyed reading this. Thanks!

John
05-14-2012, 10:40 AM
I'm actually the Dredge. Alex is the Varl.


Very glad to hear of the advancement of the develeoppement. I hope you guys have as much fun developping TBS as we will have playing it.



I take it you're the model for hte Varls then.

Leslee
05-15-2012, 02:01 PM
How does John stand up from his desk without banging his head into the ceiling???

I'm looking forward to thoroughly testing your UI. Nothing drives me more bonkers than a buggy UI! I was convinced that the creators of Hellgate: London's UI were all dyslexic!

Leslee

PS Not sure how my forum name got truncated? I guess that third letter 'e' decided to take a walk?

Wodwo
05-15-2012, 06:54 PM
I will agree with the crowds that more Technical Blogs would be great, but I'm of the opinion that I'd rather have them when there's something constructive to put into 'em, instead of on a set schedule. Having TB#2 be "Well, we wrote code. It compiled. Cool." just seems like it's taking your time away from, well, writing code (and having it compile). :P

Thank you very much for this one, however!


This is exactly what Alex from the Starfarer devs wrote on the blog when people asked for more frequent updates. And that is totally true, give us news when there are, and not the way the TV does when nothing happens.

It all depends on how John feels about it, in my opinion. I agree that a seemingly superfluous blog entry written on a schedule might use time that could be spent on other tasks, but if John (or any Dev) just wants to geek out with us for the fun of it, then I don't have a problem with it. The game will get made and it will be better if those developing it are having as much fun as possible.

So, thanks for the info, John! It's great to have an update that shows us so much! Keep doing them as long as you're enjoying it. Please don't make it into a chore, though.

Also, make sure that you have your workstation set up as ergonomically as possible. I know it'll be rough, but you need to consider your health... most things aren't designed with 11 foot tall supermen in mind.

EricStratton
05-15-2012, 07:05 PM
I'm a big git fan but I can understand the issue of size restrictions on github. We use Pivotal Tracker for our bug tracker. We like it and their staff has been very responsive. They were just purchased, tho, so I don't know how that will change things.

Please keep these tech blogs coming!

chutril
05-15-2012, 10:44 PM
I'm a programmer myself and I thank you do much for this. I love reading the coding side of the development.

PS: if you're looking for more volunteers, I wouldn't say no. I learn quickly. ;)

supamanu
05-16-2012, 03:14 AM
If I understood correctly, you're writing the core game in as3? Are you writing everything from scratch or using a framework/library? It's an interesting choice, when most people seem to be flocking towards tools such as Unity. Would love to hear what made you choose as3.

Evil Laughter
05-16-2012, 04:47 AM
Out of interest how did you choose your libraries? As an informatics researcher I'm curious how the real world makes decisions like this.

Muttley
05-16-2012, 04:57 AM
Nice update, and welcome Victor.

The more technical the blogs, the better in my opinion. :)

I occasionally use Git on Windows, but it feels like a bit of a nasty hack compared to Mercurial, so I prefer to use that for all my personal projects. I have a couple of open source projects hosted on Google Code, a couple of private ones on bitbucket, and even more I just push/pull to my Dropbox folder. :)

We also use Mercurial at work, which is handy.

How do you find Mercurial stands up with large binary blobs? I've head that neither Git nor Mercurial handle them very well.

I've also used FMOD a little (a very little, and a old version too), and found it to be a very nice library to use.

I shall look forward to the next update. :)

IanWhalen
05-16-2012, 10:37 AM
Thanks for doing this tech blog, John. Always interesting to hear about others' system architecture, toolset, processes, etc.


I am building out our DB using Amazon SimpleDB, which is a very nice key-value non-relational database. As you might tell from its name, it is pretty simple to use. Despite being non-relational and non-schema, it does support SQL query syntax.


I'm curious what were the deciding factors that made SimpleDB stand out over the other options in the key-value/non-relational/schemaless space?

John
05-16-2012, 11:55 AM
The decision to use as3 was based on a combination of 2d vector animation support, the availability of 3rd party libraries that pertain directly to the game, the ability to make native GUIs in Flash, and cross platform capabilities.

The de facto standard for creating 2d hand drawn animation is the Flash toolset. For cartoons and films you see on Cartoon Network or Nickelodeon, the scenes are pre-rendered out to video of course. Being able to take the vector animated output of Flash and render it at runtime, natively on our engine, is quite powerful.

Now, runtime rendering of vector animation is computationally expensive. For scenes with many characters animating, such as battle, we will pre-render the frames, and render them at runtime from sprite sheets. This technique will work in any engine, including Unity, although you do pay a heavy memory price. However, for many other incidental animations and scenes where the vector rendering is not a performance issue, we can choose to use the native animation format and save memory. The data structures for the frames of vector animation are far smaller than storing the rendered frames as bitmaps. For instance on the dialog scenes there should be few enough characters on the screen at once that we can get away with runtime vector rendering. This will be a big memory win, because sprite-sheeting those animations (which take up most of the screen) would consume a massive amount of memory.

Regarding 3rd party libraries, using as3isolib for our isometric scenes really speeds up development. I've written my own isometric libraries before, but I prefer in this case to just throw those out and use as3isolib. Greensock tweening library is something we use as well. The as3 libraries themselves provide a pretty rich set of functionality which we leverage. as3 has a powerful unit testing framework that works like any other xUnit implementation.

Native Flash GUIs is pretty compelling as well. Being able to craft GUIs in Flash and just run them in our engine is a powerful technique. You can use ScaleForm to accomplish this in other engines, and this a very common technique in AAA games these days. The GUI in SWTOR is Flash running in ScaleForm, for instance. Unity and ScaleForm are currently working together to make this available in Unity, but I don't know what the ETA is for that.

Adobe AIR allows you to take as3 code in the flash runtime, and compile it to native code on PC, Mac, iOS, and Android. It uses various techniques to accomplish this, including pre-compiling the as3 bytecode with LLVM and generating native code. As of last fall, you can write Adobe Native Extensions, basically a native library which you can call into from as3. So with that, the sky's the limit. Any platform-specific stuff you need to deal with (e.g. Steam, iOS GameCenter, whatever...) can be exposed through a Native Extension. This is how we are linking FMOD into the game, among other things. This is also a technique for optimization -- if something is just too slow running in as3 bytecode, we can just re-write it in C++ or Objective C, compile it natively, and away you go. I wrote a 3d engine in Java on Android a couple years ago, and the ability to just punch through to handcrafted native code (using the Android NDK) was a huge win for getting the performance I needed.

For additional platforms, such as consoles, ScaleForm comes into play for creating native applications.

All these considerations make the platform sort of a perfect fit for the particular game we are making. I am able to take quite a few technical features for granted, and focus on the game itself. The code that goes into creating our travel scenes, dialog, and combat scenes is a bespoke creation regardless of the engine chosen.

JW



If I understood correctly, you're writing the core game in as3? Are you writing everything from scratch or using a framework/library? It's an interesting choice, when most people seem to be flocking towards tools such as Unity. Would love to hear what made you choose as3.

John
05-16-2012, 11:58 AM
Mostly just ease of use in the AWS ecosystem. SimpleDB is built to interoperate easily with EC2, S3, and IAM, all of which we are using anyway. It has an https API, native APIs (in our case, Java), and supports SQL queries as well. It seems to be the simplest route for us.

JW


Thanks for doing this tech blog, John. Always interesting to hear about others' system architecture, toolset, processes, etc.



I'm curious what were the deciding factors that made SimpleDB stand out over the other options in the key-value/non-relational/schemaless space?

ZkilfinG
05-16-2012, 03:15 PM
Thanks, looking forward to more posts like this!

John
05-16-2012, 03:40 PM
Put most simply, a library needs to make my life easier, not harder.

Is it fully documented?
Is there a community of people already using it, with whom I can interact?
Does it scale up or does it fall over when stressed?
Is it cross-platform?
Is it easy to setup and get running?

If any of these questions have a negative answer, then that is a pretty major red flag. External libraries are only useful to the extent that they save you time and effort.

JW


Out of interest how did you choose your libraries? As an informatics researcher I'm curious how the real world makes decisions like this.

John
05-16-2012, 03:44 PM
I haven't stress-tested mercurial with lots of binary blobs yet. If it gets out of hand, we can always purge the history and start fresh. In practice, I rarely have to go back and retrieve things from the history. It is often better to just init a new repo from the current state and start from there. You can leave the old repo on disk for consultation if the (rare) need for revision spelunking ever arises.

JW


Nice update, and welcome Victor.

The more technical the blogs, the better in my opinion. :)

I occasionally use Git on Windows, but it feels like a bit of a nasty hack compared to Mercurial, so I prefer to use that for all my personal projects. I have a couple of open source projects hosted on Google Code, a couple of private ones on bitbucket, and even more I just push/pull to my Dropbox folder. :)

We also use Mercurial at work, which is handy.

How do you find Mercurial stands up with large binary blobs? I've head that neither Git nor Mercurial handle them very well.

I've also used FMOD a little (a very little, and a old version too), and found it to be a very nice library to use.

I shall look forward to the next update. :)

AndyG
05-16-2012, 08:59 PM
Great blog. Thank you for taking the time todo this. I am a software engineer and have been recently dabbling in game development on my own time. So it is really cool to "see" the real thing in action.

liongordy
05-16-2012, 10:21 PM
John,
This kind of information describing your efforts in producing this game should help everyone to
understand the level of professionalism you 3 are diligently working toward. You are truly making rapid advances in the progression of the "Banner Saga". Thanks for sharing this and kudos to all.

jaggers
05-25-2012, 04:51 PM
John -

*Highly* impressed that the first thing you do set up your CI/SCM infrastructure... it's unreal how much that accelerates development, especially if any of your team are distributed.

Your tools selection is pretty much best-of-breed, IMO, though be aware that Jenkins leaks like a mother; be prepared to kick the controlling node in the head periodically to recover that memory. And afterward, I strongly suggest running a manual build (I don't know why, but the way ours is set up, the first build after reboot has to be run by hand or is stays fffffed up for some reason).

Maybe that won't happen to you, but heads-up anyway!

I love this technical detail! It makes me want to drop the code I'm working on here and go work with the Stoic team ;)

John
05-27-2012, 12:55 AM
*Highly* impressed that the first thing you do set up your CI/SCM infrastructure... it's unreal how much that accelerates development, especially if any of your team are distributed.


Agreed, that really lubricates the whole development process moving forward.

Clemer
06-05-2012, 03:36 PM
This project is beautiful, and I really dig the transparency y'all are exhibiting in posts like this. Keep up the awesome work.