This is the final article in a 3-part series on the PS3 online. You can find the other parts here and here. A short follow-up has also been posted.

After exploring the good and bad of Sony’s PlayStation Network, it’s time to explore the ugly side of their online strategy: development support. This particular subject has not been made public except for a few rumblings here and there. It’s time to cut to the chase, and actually discuss the difficulties developers have encountered trying to work with Sony’s online libraries. These issues are important to highlight because they have had (and will continue to have) direct consequences on the quality of online/multiplayer games, and end-user’s experience.

PS3 Online: The Ugly

Unfortunately for Sony, we are no longer in the PS2 era, where everyone owned a PS2, and every developer had to create games for the console, if they hoped to turn a profit. Since Sony was so far ahead of the competition, it could provide terrible tools, and developers could do nothing about it. However, Microsoft has made significant inroads into the console market, and is now a very viable alternative to Sony for developers. Out of the three console makers, Microsoft provides by far the best development tools and support. Even Sony realizes this and has promised to provide better development tools this time around.

The problem remains that the PS3 is a difficult console to program due to the complex design of the hardware. Sony is a hardware company; its forte does not lie in software. As consequence, Sony’s development tools still lag far behind Microsoft’s.

To compound matters further, the Sony’s online support can only be described as poor to atrocious. Perhaps one day this will improve — after all, the console hasn’t even launched yet — but right now, the situation is dire. Sony seems to have taken the usual laissez-faire approach, which is to provide rudimentary online libraries and let individual developers figure out how they want to create lobbies, provide matchmaking abilities, track online scores and records. They are providing very few tools, and very little infrastructure.

The libraries seem to have been developed in a very ad-hoc way, with no clear goal in mind. For a long time, it seemed that developers were expected to implement every aspect of the online experience. Then Sony seemed to change their minds, and claim they would provide all the tools the developers would need. But then the promised features have rarely shown up. When they have appeared, it’s been in a very incomplete format, with the vague promise that things would be patched up in the future.

If the poor quality of the libraries wasn’t enough to hinder developers, then the repeated delays would definitely drive the nail in the coffin. Sony has been so slow in providing libraries and test hardware that it is surreal that it even expects developers to release launch games with online support. Even as recently as this summer, developers had still not received access to the Sony online test servers.

The development support in itself is already a serious issue, but it almost pales in comparison with the internal politics and friction that lies with the three major game divisions in Sony: Sony Japan (SCEI), Sony Europe (SCEE) and Sony America (SCEA). The creation of the worldwide games studios division was supposed to solve all the issues of internal divisiveness. Instead, it has done very little to stem the tide. No region seems to be clearly in charge of its own destiny. As a developer, you don’t produce a game for Sony, you produce it for one of the regions. If you release a game worldwide, then you must go through at least 3 separate QA processes. The worst problem is that each division has different strategies and different requirements, especially when it comes to online functionality.

Instead of working together to offer an online system and APIs to rival Xbox Live, they’ve each gone on their own developing their own systems, undercutting one another. Each region has a different idea of how important online support is, and whether or not games need to produce downloadable content, or have online support available at launch. For example, no Sony game can launch in North America without online support, whereas Sony Japan will accept games without it as long as they can be patched in later.

This disparity can be useful to cater to the different markets and peculiarities of each region. It becomes problematic for the developers who have to deal with conflicting demands. It just makes their lives more difficult.

The problems between different regions should not be understated. Sony Japan always has final say in any discussion, but different regions have trouble even communication with each other. Sony Japan rarely discusses anything with the two other divisions, let alone keeping them updated on what is in the pipeline. They just like to hand down unilateral decisions every once in a while. Sony Japan wants to continue to run the show, even though Sony America is a lot more experienced in online support.

As a relevant example, SCEA had apparently developed a complete online solution for the PS3 (to put it on par with Xbox Live for multiplayer support, ranking, achievements, etc). Sony Japan reportedly came in and cancelled the whole thing, simply saying, “no, you have to use the libraries we’ve put together”.

Perhaps what is most alarming is that several games scheduled for the US market have announced that they will be using Xfire as a middleware solution for their online support. The deal between Sony and Xfire comes at an extremely late stage in the run-up to the launch. It is clearly a move born out of desperation. It’s is a last minute decision to use middleware to ensure that games will launch with online support in the US, and it speaks volumes on Sony’s own online support (or lack thereof).

Xfire is simply a stand-in for the entire matchmaking and multiplayer side of the service Sony was supposed to offer (never mind that Xfire offers messaging as well). The following quote is most telling as to the gap in service in Sony’s offering.

“We support the PlayStation Network friends, matchmaking, and messaging service, and we’ve also integrated Xfire,” said Sites. “So you can use both, or you can choose to use Xfire, or just the PlayStation Network. We’ve integrated them so they work well together.”

The lack of a consistent system to offer multiplayer support will lead to vastly different levels of quality across different games. In comparison, Microsoft has only ever allowed one publisher to handle their own multiplayer service on Xbox Live: EA. And judging by how poorly consumers are responding to EA’s online support, EA is paying a steep price for its arrogant belief that they can always do better.

The route Sony has taken so far will only further fracture its online community into several smaller communities, unable to talk to one another, or interact with each other effectively.

When you speak to developers privately, they express a stunning level of frustration. Repeated delays in the delivery of online APIs, delays in coming-up with online testing kit, incomplete or missing libraries, promised features that are never delivered. The list goes on and on. They have to deal with the bloat of the OS, which eats up nearly 100 MB of memory (and one SPU), and provides very little functionality to the game. In comparison, the OS for the Xbox is rumored to only use up 3.5 MB. (Correction: an anonymous poster suggests that the Xbox reserves 32 MB for the OS. Please see the comments below.)

“At present, however, there’s no system for actually reading or responding to messages while you’re still in the game, as the operating system doesn’t take resources away from games in order to do that - however, according to Harrison, that functionality may well appear in an OS update, presumably based on whether users actually express a desire for it or not.”

It’s surprising that the OS can take so much memory, and without being able to provide a mechanism for reading or responding to messages. Another example of bloat: if you want to draw the OS keyboard in game, it will use up a further 16 MB of memory. The OS will eat up another 16 MB of memory on top of the 100 MB it’s already using up; the amount of memory used by the OS is simply ludicrous.

I personally question Sony’s goal with their online service. It comes across as a purely money grabbing effort. Of course they are a business, and will seek to make as much money as possible. However, the only bright spot is the PlayStation Store, which is hoped to become the center for millions of micro-transactions. Every thing appears to be geared around the moneymaking side of the service. Meanwhile, the actual user online experience will suffer. It is as if Sony’s management failed to see the obvious fact that a poor multiplayer experience will not generate more sales in their online store.

As to the notion that - unlike Xbox Live with its subscription-based charges- to the fact that the service is free, the reality is that you get what you pay for. The multiplayer experience on the PS3 will pale in comparison to Xbox Live. The only games that will do well are the ones using Xfire, or those where the developer has invested a lot of their time creating a fun multiplayer experience.

It seems that Sony hasn’t actually learned their lesson from their days supporting developers on the PS2. Perhaps they will only realize their mistake once they have been knocked off their perch at the top. Let’s hope it’s not too late by then.

A short follow-up piece has been posted addressing some of the main feedback.

Share and Enjoy:These icons link to social bookmarking sites where readers can share and discover new web pages.
  • del.icio.us
  • devbump
  • digg
  • Furl
  • Reddit

24 Responses to “Analysis of the PS3 Online: The Ugly”

[…] « EA Dives Into Digital Distribution Age Analysis of the PS3 Online: The Ugly » […]

[…] « Analysis of the PS3 Online Analysis of the PS3 Online: The Ugly » […]

[…] but it’s worth reading. Share this post: email it! | bookmark it! | digg it! | reddit! | kick it! | live it! Posted: Sunday, November 12, 2006 7:31 PM by Ozymandias Filed under: Online Gaming, Playstation 3,Sony Network […]

How much did microsoft payed this website to come up with all this crap…..funny/sad thing is that people will actually believe this

Shame it’s true, I really hoped Sony had learnt from the PS2.

Pechito: the articles written here are full of references and many good arguments. It’s in my mind a very well written one.

Your comment on the other hand, doesn’t contain any good counter argument, in fact, it doesn’t contain anything. Your comment is “crap”, the article is not.

[…] Μια και κάνω Sony-bashing, ας λινκάρω και στην ανάλυση του Inner Bits περί PlayStation Network. Έρχεται σε τρία μέρη. Το conclusion; As to the notion that — unlike Xbox Live with its subscription-based charges — the service is free, the reality is that you get what you pay for. The multiplayer experience on the PS3 will pale in comparison to Xbox Live. The only games that will do well are the ones using Xfire, or those where the developer has invested a lot of their time creating a fun multiplayer experience. It seems that Sony hasn’t actually learned their lesson from their days supporting developers on the PS2. Perhaps they will only realize their mistake once they have been knocked off their perch at the top. Let’s hope it’s not too late by then. […]

Thanks for the feedback guys. If you have any questions, I’ll answer the ones I can…

SONY will get it right, Rome wasn’t built in a day. The only reason M$ seems to be doing so well is because of their bat shit crazy strategy of lets lose billions of dollars every year for a decade untill the other guys are dead and then we can rape everyone.

Raoul–all three parts (yes, Sony fanboys, there are three parts, one of which is all pro-Sony) are an excellent read. You don’t talk down to the audience, you support your statements with facts (*GASP* Facts?!?! On the interwebtubes?!?!?!) and you’re not afraid to call out a company for their damaging hubris and global infighting.

It’s just too bad that certain established gaming sites (IGN, 1Up) can’t even match the quality of writing and journalism in this piece. Kudos to you, Sir.

Kamiboy, can you give one example of where Microsoft has used “their bat shit crazy strategy of lets lose billions of dollars every year for a decade untill the other guys are dead and then we can rape everyone”? Name one product that they lost billions on to eliminate their competition, and then went forth and “rape[d] everyone”?

And please don’t say Windows or Office, because those 2 pieces of software have almost been their sole breadwinners, no lost billions there. So… example please?

Thanks Matt. Much appreciated.

Slight correction: the 360 reserves 32 MB for system use. It does not actually use all of that, either - more like 24 MB. It takes more so it has room for future expansion. This is so they can change their own internals without affecting already developed games (or, to be more precise, so that developers won’t take advantage of extra system resource that may not be there in the future after an update). The fact that Sony does not do even this is a pretty ridiculous oversight, even besides the fact that they take 100 MB.

Instead of harsh comments towards MS or Sony, look at what both companies are offering you the user. It’s not about brand loyalty or what MS stands for, it’s about the services you can use and enjoy. Isn’t that why we play the games? To have fun?

Instead of feeling resentment towards a company that is actually getting it right, why don’t you take a step back and wonder why Sony isn’t stepping up the plate to make online gaming better to it’s customer base?

The article makes many good points. Bottom line is, for current console online gaming, there is only one legitimate choice right now and that’s XB Live.

Very good article, Thanks for the good read.

Thank you Mr. Anonymous. I’ve updated the article accordingly. The numbers you suggest seem a lot more reasonable.

I think Sony has done some amount of buffering for future needs, but they’ve also not coordinated well with other areas (such as the network/SCE-RT team). So to use some of the other features, you have to give the OS even more memory. Good planning.

To caveat on earlier speculation that Microsoft loses “billions” of dollars just to rape consumers in the future is pure shenanagins! Every corporation that operates in the free world does it to make money. They lose money on the hardware (as do many hardware companies) and they make that money back on software, and in Microsofts case… xbox live transactions (when dealing with xbox brand). I’m sure they are taking a hit with their new zune player as well… but they hope to make that up in music sales, accessories etc. I really want both systems to do well because competition is good. I dont want either system to totally blow out the other because if you’ve played Madden 07 for xbox 360.. you’ll realize that because of no competition it cannot do things that nfl 2k5 did 3 years ago… think about it! I’m signing off. Great story!!!

Good read, this will really influence my choice in console.

Interesting… reminds me of my past life in the corporate world. Animosities will always exist between different departments like between Ed Fries and J. Allard. This is not news.

Considering that Sony has packed so many new technologies into 1 box, and over 100 titles in development right now; it’s obvious that many activities are happening in parallel. It’s only fair that the PS3 OS is released in stages. This may not be an “Online is not important” story at all. The basic system has to work wonder first.

Even MS took over 1 year to fix (and still not done) all the nuggets in Xbox 360. The proposed XNA nice features didn’t get deliver until late this year, the hardware was (still is ?) unstable, HD-DVD playback is only added recently and touted 4.7 million lines of code (by taking *existing* code from MS’s other departments).

I think just the testing cycle within such a condensed time would have worked the product team to death (e.g., Backward compatibility, Blu-ray + Blu-ray Java + Blu-ray LIVE, XMB, all layers of gaming, basic online, WiFi, SIXAXIS, Linux booting, HDD, Hypervisor, EyeToy, advanced PSU and cooling unit, Cell and LongRun, etc. …).

Now that PS3 is launched, all eyes will be on their followup moves.
So far I have not heard of crashing PS3s due to hardware faults (unlike 360 launch boxes). They have been doing firmware update for PSP smoothly so far. For MS, I’d advise them to pay more attention to their firmware testing since every update result in complains and bricked boxes.

To put things in perspective, Sony ran a longer distance within the same time period compared to MS.

I agree with JUMPX on some of his points, especially the hardware point. However, I would like to add some comments on the other half of the equation: the software.

The significance of the developer support microsoft can provide should not be under-estimated. Microsoft has been running and leading this race for decades. There is a reason Visual Studio is the defacto codeing envirionment, and it is not because of a “monopoly”. It is a great piece of software. I am not a game developer, but I imagine that most of Sony’s games are developed using Microsoft’s software (and sony’s API’s).

Yes, XNA seems to be taking forever to release, but in reality microsoft is taking the time to do it correctly by listening to the community. They have already released the public beta (and an update), and are making changes based on developer feedback! In December, when the final version of the XNA api is released I expect many homebrew developers to port their games to the API, especially if their is a chance to cash in on xbox live arcade.

Now if we could just have Sony’s hardware and Microsoft’s developer tools and api’s we could have the best of both worlds.

Also - I want to voice my disapproval of JUMPX’s the “downplay” of the software based HD-DVD solution. *If* MS did in fact use *existing* code from MS’s other departments its a *good* thing. It would be a huge waste of resources and time if you already had the code. However, don’t forget that the 360 uses a RISC based processor… so they definatly had to re-write some code, espesically anything near the “metal”. Regardless, they must have done a good job because the avsforumer’s really seem to think it works great!

Disclosure statement: I use Visual Studio daily (not for games) and I also own a 360 and ps2.

Snapples, I am not downplaying the HD-DVD add-on. The “existing code” comment was mentioned by the developer himself (Check his blog). As you mentioned, code reuse may be a good thing. In comparison, Sony has to build the entire Blu-ray run-time stack pretty much from scratch due to a brand new CPU architecture. The 4.7 million code count is just an indication of the project scale (plus porting of J2ME to Cell).

The rewarding part is Blu-ray playback is excellent and has low CPU load (according to Kawanishi). Users are reporting very fast response time too.

The problem is they may have bitten more than they can chew for the launch. Now that PS3 is out, hopefully they have time to regroup and advance. The arrival and announcement of PS3’s single sign-on support recently may have signified a milestone passed (as much as it was late !).

From the demand side, online is certainly interesting and a huge growth area, but not as mainstream as the media painted (yet). Also, there are more than 1 way to jumpstart the user base. I don’t think Sony or Nintendo, especially Nintendo, will necessarily want to follow XBL’s model (lest they will be playing catch-up forever, and take up was slower than expected). The goldmine is probably somewhere between iTunes, Xbox Live, XFire, SecondLife, MySpace, YouTube or even Skype.

But yes, some basic game libraries need to be in-place post-haste.

@Jumpx:
You seem to be saying that it is fair for Sony to release a half done system with absolutly no support of functionality? We shouldn’t have to wait for patches to fix problems. Most developers use patches to fix something they didn’t notice. Sony cannot say they didn’t notice not finishing their console. Esspecially in the online arena.

Also, great article. It just gives me more things to push into the faces of those stupid fanboys…

This is a very good read. I was working in the game industry when Sony first broke cover. Nintendo were the 400lb Gorilla and trying to get a dev kit out of them was something of a black art. Then getting a deal that left you with a profit, well that was something else. Sony came along and gave out dev kits to anyone who wanted one. It was a breath of fresh air. That is why there are simply so many Playstation games out there. However, the dev tools for the PS1 and 2 were a pig to work with. What Microsoft are doing, which is very clever, is taking it to the next level, giving out dev kits to the world and his dog, and making their system easy to code for. I have yet to see any financial analyst commenting on the next-gen war to properly take this into account. It is hard to see Sony coming first or even second in this race, at this point in time.

This is a great read. If anything it give some insite of what the Dev are up against. Very informative.

Something to say?