The true PS3 super-computer

[alpha_dk]

alephnull said:

@ ProfCrab

I don't know what to tell you, but do you really think the number of programmers who know how to do low level programming is going up? It's not just game engines either. I went for an interview for the AIX group and they were just oozing desperation as the number of people who can do kernel programming ain't what it used to be.

The direction of the industry for years now has been RAD. Web 2.0 is all the rage. All of the undergrads here complain because they have to learn C which isn't "the road ahead". Most university CS programs don't even require C anymore.

You honestly think this doesn't have an effect on the pool of talent?

You said:

"There is not one company developing for the PS3 that has anyone that can figure it out with the money that is on the line?  Especially at Sony of all companies?"

You are right, what I was saying that NOT ONE company developing for the ps3 has a single programmer familiar with traditional parallel techniques. 

 Could it be, that instead of the number of programmers that know how to do low level programming is not where it used to be, instead of not what it used to be?  I have seen no evidence that the number of programmers who know C has gone down, and this is something that I have researched.  As the pool of 'Programmers' gets larger, I am sure the pool of 'kernel-level' programmers is becoming smaller by percent.  But rather than it being a problem with the size of the pool, couldn't it just as easily be the increased competition from projects that conform more to the stereotypical 'hacker' mindset (hacker being used in its literal sense, not its 'cracker' sense)?

 There are currently several Linux distributors, and probably all of them hire kernel developers.  Similarly, there are several BSDs (open, net, free, etc).  Companies that distribute these also need kernel hackers.  Then you have Apple needing to hire developers for the Darwin kernel (another BSD variant), MS needing to hire kernel devs for their kernel, etc.  As 'other' solutions than windows become more economically viable, there is increased competition for the programmers with the skills to work on them; AIX, being closed-source, likely has a hard time attracting some of them when there are alternative options.

 Furthermore, how do people become kernel developers?  By writing for kernels.  These days, kernels are very large, and writing from scratch isn't going to appeal to as many people.  Therefore, anyone interested in kernel development is going to work on a pre-existing kernel.  Because AIX is closed source, that means that they will likely work on something other than AIX.  These people who are learning kernel development will then be the next people headhunted by Red Hat, IBM's linux division, etc; as they already know the code, they would likely be willing to get paid to do what they are currently doing for kicks.  I feel that blaming a shrinking number of coders for AIX's inability to find hackers is a specious argument when the entire industry is still basically growing, and it can easily be explained by increased competition from Linux peddlers needing kernel devs.

I can't argue with this, but I fail to see how it applies to the argument; this has been ongoing for years, and development of low-level stuff has only increased recently (see LLVM, rate of Linux development, rate of BSD development, etc.  They are all rapidly growing)

Was this sarcasm? I think it was, but wanted to make sure because it makes no sense if not.

 In short, I don't think that you can use a single company's inability to hire for a completely unrelated position to generalize across all video game companies not having the skills to program for the cell because they have become more average over the years.  I would need to see real proof of that before I would believe it, and even then I would likely take it with a grain of salt.

The cell is hard to program for because there is no cache-coherency between SPEs. This used to be the norm back in the Super-Mini days but single chips kept increasing in clock speed and the age of super computers died.

There is more hard with programming the cell than the lack of cache coherency.  That is a thing that is hard about it, and programmers will have to get used to it, but people are not used to SIMD'izing general purpose code; we have a lot of experience with SIMDizing certain types of algorithms, but not much for others because there has not been much need to do it yet, as there have really only been special-purpose SIMD processors.  With the advent of SIMD general-purpose processors, more research is going into this, and Cell (and other similar) compilers are only going to get better as time goes on.  Which is the entire point of me saying that Sony decided to use it too early; IBM has had great success with its Cell compilers, but it is focusing on their own needs and not Sony's (as they should).  In a few years, this research will likely get into more generalized speed ups.

EDIT: Some source material for some claims i made:

Numbers of linux kernel developers has raised from 483 to 1057 in 3 years: 
http://www.linuxfoundation.org/publications/linuxkerneldevelopment.php