FOF Extractor (Updated to Version 4.3.0, 6/18/2006: Draft History Screen)

[gstelmack]

FOF Extractor Suite is a suite of utilities for writing reports from FOF that the base game does not do. It consists of the following utilities:

FOFExtractor: The original. FOF Extractor process player report windows to dump a CSV file listing details about a group of players, and can also dump scouts and coaches. The key thing about this utility is that it uses an OCR algorithm to grab data straight from the screens, which means it can be a bit flaky to run (activating ClearType in Windows gives it fits, for example) but also means it can grab data not available in any other way. Notably it gives you scout-adjusted ratings for the player.

DraftAnalyzer: Extractor's sibling, this tool runs against an Extractor dump of a draft class to allow easy analysis of the draftees. It tells you who has good combine scores, who has poor combine scores, and allows you to adjust various weights to set your own rankings.

LogFileProcessor: A recent addition, this converts game log files to CSV for help in analyzing how your gameplan is working. Can also generate an HTML table of gameplan data from a given team's point-of-view.

FOFInterrogator: Another recent addition, this utility reads the binary game data directly to dump CSV files of pretty much every internal game table. It does not dump anything that the game UI doesn't give you access to (so no actual player ratings for cheating), but its key purpose is to dump all this data in one place for easier sifting in Excel or to use in creating databases for league web sites.

FOFGameViewer: An off-line viewer for the game log files.

FOFExtender: Provides some spreadsheet views of your saved games / multiplayer leagues that aren't available in the base game. For example, it includes tables of scouts and coaches that include all of their attributes for easier browsing when trying to decide who to hire.

For those that don't know, CSV files are comma-delimited text files that are easy to open in a spreadsheet. This allows you to use Excel to sort and analyze lists of data from the game in your own customized way.

[sabotai]

Sweetness. This is one cool util.

[sjshaw]

You are a freakin' genius

[VPI97]

Damn, Skippy....

How did I miss this the first time around?

[Fritz]

someone say Skippy?

[Franklinnoble]

Originally Posted by Fritz

someone say Skippy?

I dunno, but who's "ANN"?

[fantastic flying froggies]

I am sad to say that the known issue (i.e. me !) is still not resolved in ver 1.1. I know you said you didn't fix it, but I'm just confirming it...

[Ben E Lou]

Archived.

[gstelmack]

Originally Posted by fantastic flying froggies

I am sad to say that the known issue (i.e. me !) is still not resolved in ver 1.1. I know you said you didn't fix it, but I'm just confirming it...

Making this thing support ClearType is not going to be easy. My plan for a nice, simple, elegant approach to grabbing the data from the screen completely falls apart when the fonts get blurred. Anyone know of a good open-source OCR library?

[fantastic flying froggies]

Greg, I've playing around with differents windows setting to try and fix things on my end, and I'm not so sure anymore the issue is with ClearType.

When removing the ClearType option, the extractor still crashes. Do you know of other windaws settings that would have similar conflicts with your app ?

[gstelmack]

The only other reported crash was caused by having the Extractor window on top of the player report. Making sure that the Extractor was not covering up the player report window fixed that issue.

[fantastic flying froggies]

Tried that too... still not working...

[Fritz]

Originally Posted by Franklinnoble

I dunno, but who's "ANN"?

Funny thing. Right before I haed this for the first time I had just sent a personal and somewhat flirty e-mail to woman named Ann. I read this post and thought "WTF?? Did I post my email to the FOFC?" So I started looking around so I could delete it, but with no luck. Man did I feel sick until I realized what Franklin was talking about.

[fantastic flying froggies]

what would FOF stand for then ? Fritz 's Old F*ck ???

[sabotai]

bump

[RPI-Fan]

Great utility!

Is there a way to use this to easily track changes in player ratings?

[sabotai]

RPI, what you could do is run the utility, rename the outputted file (to something like "Season 2004" and the run it again later in the seaosn or the next year. Open up both spreadsheets and find the guy(s) and then copy them to a thrid spreadsheet and there you go.

[sabotai]

Bump again (because people keep asking for a utility to do things that this does in another thread...)

[gstelmack]

Just a small note: there are currently two known bugs that I am looking into fixing:

- Combine numbers with a zero after the decimal point (i.e. 5.09) appear to be dropping the zero (appearing as 5.9). Should be an easy fix to some stupid math problem.

- There is at least one user with a non-standard font that throws the utility off. Since I only care about around a dozen characters, I hope to be able to detect and support this font.

[fantastic flying froggies]

Originally Posted by gstelmack

- There is at least one user with a non-standard font that throws the utility off. Since I only care about around a dozen characters, I hope to be able to detect and support this font.

I'm still there, Greg, and still hoping...

[gstelmack]

Originally Posted by gstelmack

Just a small note: there are currently two known bugs that I am looking into fixing:

- Combine numbers with a zero after the decimal point (i.e. 5.09) appear to be dropping the zero (appearing as 5.9). Should be an easy fix to some stupid math problem.

- There is at least one user with a non-standard font that throws the utility off. Since I only care about around a dozen characters, I hope to be able to detect and support this font.

Both are fixed. Update at http://home.nc.rr.com/gstelmack/fofextractor.htm. It still does not support ClearType, but it will now handle large fonts / 120-dpi desktop settings and fixed the stupid (on my part) decimal point error.

[gstelmack]

I have a new update available. This is strictly a trouble-shooting build. For those that are having problems, this will now tell me what field it stops on when it has trouble. I also caught one case where it was hanging up, but it will simply report an error in this case. It will also now abort when it hits an error rather than trying to continue processing (and potentially opening a bunch more error dialogs).

[3ric]

Thanks, Greg. I ran it again and it stopped at Volatility, showing the message "Could not extract detailed int".

[gstelmack]

1.1.3 is now up. It fixes my stupid bug where it would ALWAYS report "Could not extract detailed int" for a draft list. One of these days I'll stop putting stupid bugs in

[Rizon]

Originally Posted by gstelmack

1.1.3 is now up. It fixes my stupid bug where it would ALWAYS report "Could not extract detailed int" for a draft list. One of these days I'll stop putting stupid bugs in

Greg,

The 113 extractor works perfect for me when I use it on team rosters, but when I try it on the Free Agents, it stops on volatility and gives the error "Could not extract detailed int".

Also, when I try it on the Draft Preview, it works ... sort of. It gets about to the OLmen, then completely freezes. I was curious if it was system resources, so I kept the Resource Meter open, and watched it get munched. After about a minute in a half, the resources went from 90% to 0%, then froze the computer. I'm not sure what's up with that. I have 768mb RAM, and I think I might be getting punished for being too lazy to take WinME off this computer.

[gstelmack]

Originally Posted by Rizon

Greg,

The 113 extractor works perfect for me when I use it on team rosters, but when I try it on the Free Agents, it stops on volatility and gives the error "Could not extract detailed int".

Free Agent lists mess things up because there is a mix of player types (undrafted and veterans). 1.2 will be much better able to deal with this. 1.1 works best on team rosters and draft lists.

Originally Posted by Rizon

Also, when I try it on the Draft Preview, it works ... sort of. It gets about to the OLmen, then completely freezes. I was curious if it was system resources, so I kept the Resource Meter open, and watched it get munched. After about a minute in a half, the resources went from 90% to 0%, then froze the computer. I'm not sure what's up with that. I have 768mb RAM, and I think I might be getting punished for being too lazy to take WinME off this computer.

You might. Win9x is really bad about resources. I'll double-check to make sure I'm freeing everything (I'm using bitmaps to get screenshots, and that is likely what's leaking on you), but at first glance this may be a problem with ME not releasing freed resources in a timely manner.

[Rizon]

Originally Posted by gstelmack

... may be a problem with ME not releasing freed resources in a timely manner.

After going through all the possible ID-10-t user errors, I kind of figured that's what it might be. I could really build a time machine to go back and kick myself.

[gstelmack]

Version 1.2 is now available at http://home.nc.rr.com/gstelmack/fofextractor.htm. See details on the site and in the readme, but the big thing is that 1.2 has an improved OCR initialization structure that handles big fonts automatically (tested with 96 DPI and 120 DPI), lets me pull in more data, and makes it easier to handle mixed lists of veteran players and rookies that have not been through a training camp. It is also generally more robust: you are now far more likely to see a "0" in the data than have the program freeze on a player.

Because of the under-the-covers changes, and because dealing with a font like Arial means it's hard to programmatically determine how wide a space is, or if the letter is a capital-eye or lowercase-el, this should be treated like a beta release. Let me know of any problems, and I welcome feedback on the layout of the new columns, especially in lists of mixed players. I'm considering options to eliminate columns (for example, tell extractor not to dump combine numbers even if the free agent list has draft players), and I'm also willing to re-arrange columns, add flag columns to indicate the existance of other columns, write out column headers, etc. I am also reading in all the data for all players, then spitting it out to a file, so I can calculate column values or add summary data as desired (if better to have here than in a spreadsheet). So feel free to make suggestions that would make the data file more useful and more user-friendly.

[3ric]

Thanks, Greg! I'll make sure to check it out and get back to you if I find anything.

[gstelmack]

1.2.1 available, primarily with some new debugging aids. There is now a "Debug First Player" checkbox that will write the screen captures to PlayerInfo1.bmp, PlayerInfo2.bmp, and ScoutingReport.bmp, plus a FontData.txt file with some sizing information on the font being used. These can be sent to me in cases where the screenshot does not make it clear what's going on.

I've also put code in to place the dialog in the upper-left corner to help reduce the chance that Extractor is blocking the PlayerReport window.

[gstelmack]

1.2.2 available. Fixed a bug that caused the "Current Impression" field of the draft list to extract as 0. Capital-eye and lowercase-el got confused on me.

[3ric]

Thanks... I did a draft list before that was affected by that bug, now I can fix it...

[3ric]

Greg, any chance of making it possible to export a list of the available coaches and scouts?

[gstelmack]

Originally Posted by 3ric

Greg, any chance of making it possible to export a list of the available coaches and scouts?

Oooo, good idea. Yes, it's certainly feasible. I have one other project I'm working on (probably won't release it unless I can figure out how to move my draftee-ranking-formula to an editable text file so I don't give away any trade secreats ), and then I may just have to look into this...

[gottimd]

I am using this Mod, and adding onto it and I hope to be done within a week or two. As of now, I just have it working on the current rosters, which I am trying to complete and once I do that I will begin working on the Draft Extract.

What it will do is, once you extract the data using the FOF Extract Mod, you copy and paste the data into the spreadsheet that I will provide in excel. Press a button (A macro) that will parse the data and put all qualities and characteristics in the appropriate columns with a column that I added named "Potential Reached". You can sort the data by position. It will make this more readable and give you the percentages and statistics (current and potential ratings) by position in a more clear format, which will be very easy to read. This will make it easier to compare your players on your team by each category in excel format.

I am hoping to build on this at a later time (Salaries,Analysis on whether to re-sign/negotiate/release a player), and since I am working in excel, I can pass this off as actual "workload" at my job. Of course my job has some work in it as well, so the development may take longer than I expect.

[fantastic flying froggies]

Excellent news guys, keep 'em coming !

[gottimd]

Wanted someone to test out and let me know any bugs for this first edition Mod. Follow the instructions, and make sure all FOF 2004 files were saved in the default directory.

Press Here for "Gottimd Extractor Plus Beta Version"

[gottimd]

Right click on the link and save as if the link doesnt work.....

Dont change the file name, there are macros running off of it....

Any suggestions would help, so feel free to let me know if I am wasting my time or if there are any features that should be added.....

[sovereignstar]

Anyone else having problems with this?

[fantastic flying froggies]

gottimd, you may want to include a readme file giving detailed instructions.

OK, I have my extractor text file. I select all, copy and paste in the FOF2004 worksheet of your spreadsheet.

What do I do next ? I've tried running a macro, but excel doesn't find any.
Any suggestions ?

[sovereignstar]

Originally Posted by fantastic flying froggies

gottimd, you may want to include a readme file giving detailed instructions.

OK, I have my extractor text file. I select all, copy and paste in the FOF2004 worksheet of your spreadsheet.

What do I do next ? I've tried running a macro, but excel doesn't find any.
Any suggestions ?

Exactly what he said.

[gottimd]

On the first sheet, I think it is called "Main", there are instructions and buttons. Sorry I would have responded earlier, I was out of the state with no access. The macro's should work, I have tested a few times. I am still working on it though. THe only reason I can see that it wouldnt work is due to O/S differences. I use XP.

Try it again....

[fantastic flying froggies]

Originally Posted by gottimd

On the first sheet, I think it is called "Main", there are instructions and buttons. Sorry I would have responded earlier, I was out of the state with no access. The macro's should work, I have tested a few times. I am still working on it though. THe only reason I can see that it wouldnt work is due to O/S differences. I use XP.

Try it again....

OK, 1st problem : there is only one sheet, called FOF2004 and it's completely blank.
Maybe try reposting your excel file ?

[gottimd]

Maybe something is wrong with how it is hosted. When I download from the site, everything works. Anyways, would someone be able to Host it on their site, or at least check it out to see if it works? Let me know, and I will email you the file.

[fantastic flying froggies]

I can't host your file as i don't have any webspace, but if you want, you can email the file and I'll tell you if it works.
Then we'll know for sure if it's your file or the web host.

I'll PM you my email adress right now.

[gottimd]

Got it! Just sent you the Excel file.

[fantastic flying froggies]

Originally Posted by gottimd

Got it! Just sent you the Excel file.

Got it, it works fine ! And it's a beauty...

You've just saved me hours of painful importing and formatting !

Some comments :
- works great for roster dumps, not so much for draft preview dumps
- add in the possibilty to change filepath and/or filename

Obviously, your liink is broken as the file you emailed works fine.

That's it for now. Again, great work and thanks for sharing...

[gottimd]

After I complete this one, I want to start working on a Draft File. I have updated what you have to version B. This adds in the ability to compare 3 players of any position on the main page. Also, at the bottom, as of now, it gives you the averages of each position (Characteristics such as age, volatility, Intelligence, etc) and the amount of players you have at that position. I will email it to you shortly.

I am not sure how to make the paths static based upon user, so I made it set to the default as noted in the directions.

[fantastic flying froggies]

Great.

You've got an error in version b. It gives an out of range error.

It's easy to fix, with the debugger. I did it myself. You have a reference to FOF2004.xls where your new file is FOF2004vb.xls.

[gottimd]

Oops, I renamed the file without changing the VBA, good catch.