CFB 25 COMPLETE Roster and Ability Database now Online!

[jtrosclair]

Hey guys!

I have a HUGE contribution to report to the community in that I've scraped all of the rosters as of the V3 update. Give it a check out at: https://www.teamcrafters.net/app/players

In case you were wondering, no, this was not just scraping an API from EA. Their API only has the top 100 players and nothing more, trust me, I tried.

Backstory: the ratings/bios were grabbed via an automated PS5 script and Chiaki using OCR to gather the ratings, As you will see it will be confused in some characters like the names and bio fields, but the ratings and abilities look to be in really good shape.

This process took 3 weeks of leaving my PS5 on overnight, and various fine-tuning to the script as this is my first attempt with machine vision and data collection.

Errors in the data can be reported to me in the TeamCrafters Discord server, but I know once Roster V4 drops I should be able to get it out quicker with some performance improvements for the data collection. But since EA doesn't just tell us the rating changes, it will have to be re-collected from scratch...

[UtahUtes32]

Quote: Originally Posted by jtrosclair

Hey guys! I have a HUGE contribution to report to the community in that I've scraped all of the rosters as of the V3 update. Give it a check out at: https://www.teamcrafters.net/app/players In case you were wondering, no, this was not just scraping an API from EA. Their API only has the top 100 players and nothing more, trust me, I tried. Backstory: the ratings/bios were grabbed via an automated PS5 script and Chiaki using OCR to gather the ratings, As you will see it will be confused in some characters like the names and bio fields, but the ratings and abilities look to be in really good shape. This process took 3 weeks of leaving my PS5 on overnight, and various fine-tuning to the script as this is my first attempt with machine vision and data collection. Errors in the data can be reported to me in the TeamCrafters Discord server, but I know once Roster V4 drops I should be able to get it out quicker with some performance improvements for the data collection. But since EA doesn't just tell us the rating changes, it will have to be re-collected from scratch...

Cool!

I use Chiaki, but want to know more about:

"...the ratings/bios were grabbed via an automated PS5 script and Chiaki using OCR..."

[jtrosclair]

Quote: Originally Posted by UtahUtes32

Cool! I use Chiaki, but want to know more about: "...the ratings/bios were grabbed via an automated PS5 script and Chiaki using OCR..."

I'll leave this pretty high level

Chiaki is an open source PS remote play client that allows you to use keyboard inputs.

OCR (optical character recognition) is machine vision that reads text that is on the screen.

I set up my PS5 using Chiaki, connected to OBS, used virtual camera to feed in a python script that used OCR to read every roster entry in the game. I added logic to flip through the pages and move through the grid.

Many, MANY, hours of trial and error.

[K22]

Quote: Originally Posted by jtrosclair

Hey guys! I have a HUGE contribution to report to the community in that I've scraped all of the rosters as of the V3 update. Give it a check out at: https://www.teamcrafters.net/app/players In case you were wondering, no, this was not just scraping an API from EA. Their API only has the top 100 players and nothing more, trust me, I tried. Backstory: the ratings/bios were grabbed via an automated PS5 script and Chiaki using OCR to gather the ratings, As you will see it will be confused in some characters like the names and bio fields, but the ratings and abilities look to be in really good shape. This process took 3 weeks of leaving my PS5 on overnight, and various fine-tuning to the script as this is my first attempt with machine vision and data collection. Errors in the data can be reported to me in the TeamCrafters Discord server, but I know once Roster V4 drops I should be able to get it out quicker with some performance improvements for the data collection. But since EA doesn't just tell us the rating changes, it will have to be re-collected from scratch...

So very cool....thanks a lot

[UtahUtes32]

Quote: Originally Posted by jtrosclair

I'll leave this pretty high level Chiaki is an open source PS remote play client that allows you to use keyboard inputs. OCR (optical character recognition) is machine vision that reads text that is on the screen. I set up my PS5 using Chiaki, connected to OBS, used virtual camera to feed in a python script that used OCR to read every roster entry in the game. I added logic to flip through the pages and move through the grid. Many, MANY, hours of trial and error.

Wow!! That's awesome.

[ChaseB]

This is super cool jtro, I'll likely feature this on the front page at some point soon once I poke around a bit with it first.

Can you tell me how the TeamBuilder teams are getting on there? I assume they're real squads.

[americowins]

great work! are you keeping this updated weekly?

[UtahUtes32]

Quote: Originally Posted by americowins

great work! are you keeping this updated weekly?

There is nothing to update weekly.

When new roster updates come out, it's eventually updated on the OP's website, but we've only had 4 roster updates since July.