Hey Arc. Last year I remember you said that updating the league takes you up to 3 hours weekly.

Is that still so? Because I made a "little" (it's actually pretty big) windows app, which updates my trophies and creates juicy stats of them automatically, from the HTML source of psnprofiles.com.

If you're interested, I could develop a windows app for you which creates all the stats and lists you need for this trophy league - we could also expand it with other stats, graphs and shit.

You would need to insert the users and the plat's difficulties manually, the rest would update automatically- as long as the users don't disable showing off their trophies on public.

You would also need to install some free software (SQL Server Express) and follow some instructions to install and update the app, but we could work that out.

Also, I could take over the thread if you want and update - and enhance it - with my method.

What do you (and the other guys) think?

Pics of my app will follow...

All Games

bilder kostenlos hochladen

Days/Hours

gratis bilder hochladen

Months

bilder hochladen kostenlos

Years

bild upload

Trophy Log

picupload

@Arcturus

Thx for update. Out of curiousity, how many stars you have for BBCPE and Raiden?

BBCPE is rated as a 3-star and Raiden is rated as a 5-star.

I see, thank you for the hard work.

Wow, this is really interesting! And I have so many questions. But first, to answer your question:

Yes, I think it was a little over a year ago when I said it was taking upwards of 3 hours to update the league each week. This was back when collecting everyone's trophy data was done manually. I've since created a basic script to collect and organize everyone's trophy data automatically. However, some parts of the weekly update still require some manual work. The most time consuming part of the update is determining which new platinum trophies have been earned. I have to manually go through people's profiles on psnprofiles and locate the new platinum trophy. If this is the first time this platinum trophy has been earned, then I have to manually look up the difficulty of it from playstationtrophies.org.

As for my questions:
- can your app easily be scaled to track over 100 users?
- how easy would it be to create custom 'reports' from this app and data? (e.g. creating the platinum leaderboard, session leaderboard, busiest member, etc)
- how do you get around the CAPTCHA on psnprofiles? I can see this being an issue when updating the trophy stats for 100+ users
- what sort of enchancements would you make to this thread if your application was used?

Looks really interesting! Thanks.

I just need you to send a friend request to either vgchartz1 or vgchartz2 on PSN.

- can your app easily be scaled to track over 100 users?

I would create a new app from scratch. Of course, I already have the code to retreive the trophy data from psnprofiles and I'll be able to recycle a lot of code and sql scripts for stats too. So let that be my worry, I'd develop it with pleasure. As I said, you'll only have to insert new users and new plat difficulties.

I can make a script to insert all the current users and if you have one list of all your plat difficulties I can insert those too before you start.

- how easy would it be to create custom 'reports' from this app and data? (e.g. creating the platinum leaderboard, session leaderboard, busiest member, etc)

It depends on what you need. Exporting a list to excel that looks like the ones you post here is pretty easy. You'll only need to select the type of list and push one button to export

- how do you get around the CAPTCHA on psnprofiles? I can see this being an issue when updating the trophy stats for 100+ users

Hm, my app wouldn't cause an update on psnprofiles for each account. It just opens the psnprofile page of every user, for example http://psnprofiles.com/GhettoGlamour, and collects all the data from there (HTML source code). As far as I know, as long as the user doesn't make his trophies private, psnprofiles updates every account daily.

If psnprofile doesn't know a certain user, I think everybody can create a profile of that account (I've tried it out with two guys from our league, sorry for that)

You - or me - would have to do add those manually on psnprofiles.

- what sort of enchancements would you make to this thread if your application was used?

Look at the kinda stats I have. I could add some interesting columns to the leaderboards, like % of completion, % platinumed, % of days with earned trophies, etc.

Another thing I'd like would be a "hot games" list where you post which games have been played most per week/month/year/all time, showing how many users, trophies, points, plats,etc. they got in our league.

Another interesting stat would be how many  trophies, points, plats, etc. the whole league gets weekly/monthly/ etc. globally. I could create any list you'd like. What and how often do you wanna post any of these is up to you, of course. Maybe we could start some global challenges like: "let's get 50 plats till end of month"

Maybe you wanna PM me your skype ID so we can talk more comfortably.

How you plan to play so much
I want some recommendations

- Pulling data from psnprofiles

I haven't looked into this in months, but at the time there were a few hurdles with pulling trophy data from psnprofiles.
1. If you don't force the site to perform an update (by going through the motion on the main page and entering in the CAPTCHA), then updates are only automatically done once a week. If you go to my profile without forcing an update, you see the following below the trophy milestones section:

This would make it difficult to gather trophy data at a single point in time as automatic profile updates would be staggered throughout the week.

2. If you haven't forced a profile update in a while, eventually the site will stop automatically updating the profile. Here is a screen capture of a friend's profile:

This means that you would have to force profile updates every once in a while.

3. CAPTCHA. How would an automatted tool overcome this without resorting to forcing updates manually.

I looked into automatically gathering data from psnprofiles when I was creating the script I currently use, but stopped because of the above reasons. At the moment, I have the scripts automatically gather trophy data from the friends lists of the VGChartz1 and VGChartz2 PSN accounts via playstation.com

- As for the enchanments to the league

I would love to see all of these enchancements! What always prevents me from going further is the amount of effort needed each week to perform the updates. If more tasks were automated with scripts and such, then I'd have more time to incorporate new tables and stats. My knowledge is pretty limited when it comes to scripting and programming. I taught myself just enough Python to piece together the script I currently use to gather trophy data. The rest of the currently automated tasks (generation of Platinum Leaderboard, Busiest Members, etc) was put together in Excel with VBA scripts, which I briefly learned about 10+ years ago.