FOF Extractor (Updated to Version 4.3.0, 6/18/2006: Draft History Screen)

[Coder]

Originally Posted by gstelmack

I suck at art. I mean, did you look at the default logos I included? However, there is full documentation (which isn't much) on what format the endzone and midfield logos need to be for the game to use them. So any intrepid artist could create logo packs for any league they wanted to (hint hint nudge nudge).

I'll investigate where I'm pulling the city abbreviations from. Probably not grabbing the correct universe versions or something.

Well the old FOFViewer is using helmets from whichever league you're using. Alternatively, use the default helmets?

[gstelmack]

Originally Posted by Coder

Well the old FOFViewer is using helmets from whichever league you're using. Alternatively, use the default helmets?

No it's not. It's using whatever helmets you put in its FOF_logos folder. I guess I could use helmets from the FOF directory and put them in the endzone, but those aren't really endzone or midfield logos. And I have no real room to put them elsewhere in the UI.

[gottimd]

Originally Posted by gstelmack

No it's not. It's using whatever helmets you put in its FOF_logos folder. I guess I could use helmets from the FOF directory and put them in the endzone, but those aren't really endzone or midfield logos. And I have no real room to put them elsewhere in the UI.

Is there a place to put a league logo? Also some sort of logo for playoff games depending on Conference and round of playoffs? Not that important, but would be nice to have.

[gstelmack]

Originally Posted by gottimd

Is there a place to put a league logo? Also some sort of logo for playoff games depending on Conference and round of playoffs? Not that important, but would be nice to have.

Not yet, but there could be Not a bad idea for 4.0.3, as I already know which games are playoff games. I don't know which conference, though, so the likelihood is I'd just use the league logo for all playoff games.

[Coder]

Ran this week's IFL-games in the viewer and here are some further points:

Colors
I never thought they were going to be an issue but it's strange that you've grown so used them in the old FOFViewer. I loved seeing the green in the 'Neers when we were in possession. Is it possible to have a "league config" file? Not sure exactly how, but it would definately add to the immersion factor. I'm talking about the play-by-play now, just like in FM/CM, having the background in the main-color and the foreground in the secondary color.

Stats

I miss being able to see "running" stats for passingyards and rushing yards etc (i.e. stats being updated during gameplay). I'm guessing that's what the box-score etc is supposed to be about, but at least for the last game I watched I didn't get any stats at all.

Larger Display of Key items

I'd like to see that downs and the gameclock in a slightly larger font in a more suitable place, preferrably slightly above the field, along with the current gamescore.. sort of like an NFL-scoreboard. I can't tell you exactly why this would make more sense, it's just that it feels better to see that above the field rather than below. Could be just me though, but at least a larger font would make sense.

Summary

This is supposed to be constructive criticism. The viewer so far is awesome! I love seeing the "current drive" info, as do I like being able to see the "running" commentary in the lower right area in case I missed a certain comment.

[gstelmack]

Originally Posted by Coder

Ran this week's IFL-games in the viewer and here are some further points:

Colors
I never thought they were going to be an issue but it's strange that you've grown so used them in the old FOFViewer. I loved seeing the green in the 'Neers when we were in possession. Is it possible to have a "league config" file? Not sure exactly how, but it would definately add to the immersion factor. I'm talking about the play-by-play now, just like in FM/CM, having the background in the main-color and the foreground in the secondary color.

This is certainly possible if there is a desire for it.

Originally Posted by Coder

Stats

I miss being able to see "running" stats for passingyards and rushing yards etc (i.e. stats being updated during gameplay). I'm guessing that's what the box-score etc is supposed to be about, but at least for the last game I watched I didn't get any stats at all.

Working on it. There's a fair amount of work here to make sure penalties are correctly accounted for, but it is my current task.

Originally Posted by Coder

Larger Display of Key items

I'd like to see that downs and the gameclock in a slightly larger font in a more suitable place, preferrably slightly above the field, along with the current gamescore.. sort of like an NFL-scoreboard. I can't tell you exactly why this would make more sense, it's just that it feels better to see that above the field rather than below. Could be just me though, but at least a larger font would make sense.

All suggestions welcome on this. I'm on about my fifth iteration of that bar trying to get all the key info in and make it work.

[gstelmack]

Changes include:

• Interrogator correctly handles custom city lists.
• Game Viewer now starts each game paused.
• Game Viewer now tracks a bunch of stats.
• Game Viewer now animates the ball during each play.
• LogFileProcessor now shows progress as it parses each selected log.
• LogFileProcessor now has a "Summary" option that dumps a summary.csv file with some collected information about each game. This is used for occasional statistical checks, such as what impact weather has on each game (which prompted this feature).

I'll be collecting / fixing bug reports (especially on the Game Viewer stats) for the next week or two, then look into some visual look improvements for Game Viewer (re-arranging UI elements, adding formations to the field). Key things to look for in the stats are mis-handling of penalties (did I negate a play that shouldn't have been, or allow a play that should have been negated?), and any plays that aren't credited. Note that the GameViewer now writes a "ParseErrors.txt" file to the installation directory with info about lines I don't recognize; I'll ask for this file if you report a mishandling of a play.

[Subby]

Hey Greg - first off - great work - much thanks for your continued work on these utilities. They make MP life a ton easier.

Secondly - is there an interrogator file that maps team IDs to team cities or nicknames?

Thirdly - I can run the log file processor, but the progress bar stops at full and just hangs there - should I be seeing something different?

Thanks a ton.

[gstelmack]

Originally Posted by Chris

Secondly - is there an interrogator file that maps team IDs to team cities or nicknames?

Can't remember off the top of my head, I'd have to check. I'm pretty sure the team list has the city index, just don't remember if I also map to nickname or not.

Originally Posted by Chris

Thirdly - I can run the log file processor, but the progress bar stops at full and just hangs there - should I be seeing something different?

I'm pretty sure that means it's finished. If you click the close box and it shuts down, it was done. I'll try to add a better indicator that it's finished.

[Subby]

Excellent - it processed.

This is an amazingly powerful tool - looking forward to putting it to use.

One thought -

In the log file parser .csv, receiver set shows "Normal" when a team runs single back - even though by its nature, single back is a 3 wr set...not sure if there is anyway to change the way that is denoted (if I wanted to see success for all plays running 3 - 5 wr and sorted on receiver sets, I would miss the single back formations).

Anyway - going back now to chart last season and see what worked and what didnt

[Subby]

Originally Posted by gstelmack

Can't remember off the top of my head, I'd have to check. I'm pretty sure the team list has the city index, just don't remember if I also map to nickname or not.

I think the way the teams are sorted corresponds to their ID numbers (0-31). The export is just missing that field.

Same with cities - no id field, but I am pretty sure they are sorted according to id (0-168).

[gstelmack]

Originally Posted by Chris

I think the way the teams are sorted corresponds to their ID numbers (0-31). The export is just missing that field.

Same with cities - no id field, but I am pretty sure they are sorted according to id (0-168).

That is correct. The field would be redundant. And the TeamData.csv file has the team nickname and the city index into the city table, so you have all the info you need.

[gstelmack]

Originally Posted by Chris

In the log file parser .csv, receiver set shows "Normal" when a team runs single back - even though by its nature, single back is a 3 wr set...not sure if there is anyway to change the way that is denoted (if I wanted to see success for all plays running 3 - 5 wr and sorted on receiver sets, I would miss the single back formations).

I'm just reporting how FOF reports those formations. So it would be a Normal receiver set for the Single back formations. Receiver set can include things like "Slot" for a two-TE set where the receiver moves to the slot position. It's how the receivers line up, not the count.

[Subby]

Musings...

- It's too bad there is no mentor field in data...

- It would be nice if the player file included retired players (the scout and staff game tables seem to retain that info)...that way it would be really easy to produce reports to show every player a team had drafted...

- A feature that turned parsed birthday information and turned it into zodiac sign would be awesome...

- I wonder if agents stay the same throughout the life of the game...

Anyway...good stuff.

[gstelmack]

Originally Posted by Chris

- It would be nice if the player file included retired players (the scout and staff game tables seem to retain that info)...that way it would be really easy to produce reports to show every player a team had drafted...

Yeah, the one table is currently active players, the other is guys who have ever played in the lague. So the retirees show in one but not the other.

Originally Posted by Chris

- A feature that turned parsed birthday information and turned it into zodiac sign would be awesome...

Well, the whole point behind this is to make the data available to other apps, so you can always do an Excel table or add it to your database app or whatever.

Originally Posted by Chris

- I wonder if agents stay the same throughout the life of the game...

No clue, haven't played a career for long enough.

[sovereignstar]

Have I told you lately I love you? The Interrogator kicks ass, Greg. Good work.

[gottimd]

This is the best invention since the FOF Extractor 4.0.2.

Seriously, great work yet again. Gotta love the "FOF EXTRACTOR UPDATE" thread when you log in to FOFC. Next best thing other than "NEW SOLECISMIC GAME".

[gstelmack]

Changes include:

• LogFileProcessor now indicates when it is finished processing.
• GameViewer now handles a few more penalties:
  • Illegal Chop Block
  • Ineligible Receiver Downfield
• Fixed determination of which team the penalty was on in GameViewer (which led to some improperly negated plays for stat collection).
• Fixed some GameViewer issues with determining punts inside the 20.
• GameViewer now treats kneeldowns as rushing attempts.
• GameViewer now handles stats for the final play of the game.
• GameViewer has a modified layout. Comments welcome.

I am especially interested in comments on the modifications to the layout. Especially moving the play text on to the field; I'm on the fence about this, as I like having the current play all laid out, but it may take up too much room.

[twothree]

Originally Posted by gstelmack

No clue, haven't played a career for long enough.

Agents do change. I have had at most one agent "retire" in any given year, and a new agent shows up to take their place. It occurs maybe once every four years on average; however, I seem to get long stretches (5-8 years) with no new agents, and then back to back seasons with new agents.

[Subby]

Didn't the extractor used to pull a player ratings range for draftees? I am getting all of their general info, but no ranges like in the past...just zeroes.

[Tasan]

argh...this is the first time I've used this program and I can't seem to open a draft file. I used extractor on the draft preview and have the FOFRoster.txt When I try to open that in FOF Draft Analyzer, it then opens the TCY HTML Stats files screen, where I'm told to press cancel if I don't have these, so I do. The program just jumps back to an empty FOF Draft Analyzer screen at that point. Its all I can get. I've gone through this thread and can't find something like this, I'm sure its operator error and I just need a little help.

[gstelmack]

Originally Posted by Chris

Didn't the extractor used to pull a player ratings range for draftees? I am getting all of their general info, but no ranges like in the past...just zeroes.

Yeah, it should still work. Usually when Extractor dumps bad data files, it's because something interfered with the screen captures it uses. Did you have something covering the Player Report window?

[gstelmack]

Originally Posted by Tasan

argh...this is the first time I've used this program and I can't seem to open a draft file. I used extractor on the draft preview and have the FOFRoster.txt When I try to open that in FOF Draft Analyzer, it then opens the TCY HTML Stats files screen, where I'm told to press cancel if I don't have these, so I do. The program just jumps back to an empty FOF Draft Analyzer screen at that point. Its all I can get. I've gone through this thread and can't find something like this, I'm sure its operator error and I just need a little help.

You can e-mail me the FOFRoster.txt file and I can look and see if it's any good.

When you ran Extractor, did you double-click the first player in the draft preview window first? If not, there wouldn't have been any players for it to extract...

[Coder]

Originally Posted by Chris

Didn't the extractor used to pull a player ratings range for draftees? I am getting all of their general info, but no ranges like in the past...just zeroes.

Maybe your OS is set to cleartype?

[gottimd]

Originally Posted by gstelmack

• GameViewer has a modified layout. Comments welcome.

I am especially interested in comments on the modifications to the layout. Especially moving the play text on to the field; I'm on the fence about this, as I like having the current play all laid out, but it may take up too much room.

Not sure how I like the text of the play on the field. Is there a way to have that option turned off or make that option turned off?

Maybe, add a "Last Play" or "Current Play" text box underneath the score, but not in the field that shows the text of the play.

Also, just a suggestion, not sure how hard it would be to split the team stats up, rather than both on the same place. And one last thing (sorry), and export to CSV function for all of the game stats.

Even without the stuff that I suggest, this is a great tool!

[Subby]

Originally Posted by Coder

Maybe your OS is set to cleartype?

Nahh - I got some numbers, just not the ratings...

Also - I ran a test on the first 10 players, nothing was obstructed and still got zeroes for the ratings...

I upgraded .net v2 beta, so we'll see if that makes a difference.

[Tasan]

Originally Posted by gstelmack

You can e-mail me the FOFRoster.txt file and I can look and see if it's any good.

When you ran Extractor, did you double-click the first player in the draft preview window first? If not, there wouldn't have been any players for it to extract...

Yeah, I had the first player open when I ran it. It seemed to process through all the players. I opened up the text file and and saw all the players entries in there, or so it looked like. Just now though, I opened it back up, and now that file is empty. The file itself is still there, but its now 0 bytes and no players exist. I'll run the extrator again and see what happens.

[Tasan]

Dola

Got it working the second time through...I have no idea where the first set of data went, but it was there, and then it wasn't.

[gstelmack]

Originally Posted by Chris

Nahh - I got some numbers, just not the ratings...

Also - I ran a test on the first 10 players, nothing was obstructed and still got zeroes for the ratings...

I upgraded .net v2 beta, so we'll see if that makes a difference.

ClearType shouldn't affect the draftee ratings, as it's checking the color of the bars. Is there a possibility your bars aren't blue? Some weird color scheme? If you want to e-mail me a screenshot of a draftee I might be able to see what is going on.

.NET won't impact Extractor as it is all C++/MFC code, as is Draft Analyzer. Interrogator / LogFileProcessor / GameViewer are the C#/.NET apps.

[gstelmack]

Originally Posted by gottimd

Not sure how I like the text of the play on the field. Is there a way to have that option turned off or make that option turned off?

Maybe, add a "Last Play" or "Current Play" text box underneath the score, but not in the field that shows the text of the play.

I've got a couple of options I'm mulling over. Likely is to go back to just one line of text that cycles, and then either keeping two lines on the field for Offensive/Defensive formation, or going with my original idea of actually drawing the formations on the field. Not sure which yet.

Originally Posted by gottimd

Also, just a suggestion, not sure how hard it would be to split the team stats up, rather than both on the same place. And one last thing (sorry), and export to CSV function for all of the game stats.

Debating splitting the team stats up. I've considered coloring the lines differently based on the team, or having a separate box score entry. I kind of like knowing who the leading rusher is, though.

As for CSV export, maybe. Better use would be a box score to CSV converter, but also note that Interrogator already dumps complete stats for every game ever played in your FOF league, so anything here would be redundant (aside from the fact that the one game would be compiled in one place).

[gstelmack]

I felt so bad about the abomination that was putting the play lines on the field that I fixed it as quickly as I could. Changes include:

• GameViewer has another modified layout. Comments welcome.
• As part of the new layout, added a new menu item "Options" to GameViewer. Right now it only has one entry: "Formations on Field". If checked this will cause the play text to skip offensive / defensive formations, and these will appear on the field instead.
• Fixed a GameViewer issue with no team for a receiver who fumbles the ball.
• GameViewer now colors the stat lines for the away team with a light gray background to differentiate them from the home team stats.

[gottimd]

Originally Posted by gstelmack

I felt so bad about the abomination that was putting the play lines on the field that I fixed it as quickly as I could. Changes include:

• GameViewer has another modified layout. Comments welcome.
• As part of the new layout, added a new menu item "Options" to GameViewer. Right now it only has one entry: "Formations on Field". If checked this will cause the play text to skip offensive / defensive formations, and these will appear on the field instead.
• Fixed a GameViewer issue with no team for a receiver who fumbles the ball.
• GameViewer now colors the stat lines for the away team with a light gray background to differentiate them from the home team stats.

Nice, me likey! One cosmetic thing I would do is, where you have the attendance and weather, and the current drive, put those in the middle. Like center a "text/input" box right underneath the quarter/time/down and put the Attendance and weather and right below that a text box that has the current drive. You could make the font smaller for those.

Also, like the color difference in the stats and the blue line to indicate the drive start.

[gstelmack]

Originally Posted by gottimd

Nice, me likey! One cosmetic thing I would do is, where you have the attendance and weather, and the current drive, put those in the middle. Like center a "text/input" box right underneath the quarter/time/down and put the Attendance and weather and right below that a text box that has the current drive. You could make the font smaller for those.

I'm trying to keep the total window size small enough to fit on a 1024x768 desktop, so I don't have much more room to go vertical. I used most of it up putting the scoreboard bits at the top, which was a requested location for them. At some point I'm going to give up on having a full realistic field and chop it vertically to fit more stuff in...

[Subby]

Greg (or anyone) - have you taken a look at the Draft Order output from Interrogator? I have run it a few times at various pre-draft stages this preseason and it is always wrong - completely off...all other Interrogator csv exports are fine...

Thanks.

[gottimd]

Originally Posted by Chris

Greg (or anyone) - have you taken a look at the Draft Order output from Interrogator? I have run it a few times at various pre-draft stages this preseason and it is always wrong - completely off...all other Interrogator csv exports are fine...

Thanks.

Did you change the order manually to reflect what your leagues order is for that year?

[gstelmack]

Originally Posted by Chris

Greg (or anyone) - have you taken a look at the Draft Order output from Interrogator? I have run it a few times at various pre-draft stages this preseason and it is always wrong - completely off...all other Interrogator csv exports are fine...

Thanks.

I did twice. Once it looked off, once it was correct. I'll have to double-check again.

[gstelmack]

Originally Posted by Chris

Greg (or anyone) - have you taken a look at the Draft Order output from Interrogator? I have run it a few times at various pre-draft stages this preseason and it is always wrong - completely off...all other Interrogator csv exports are fine...

Thanks.

Okay, looks like it is off by a year. I think the first block marks what picks have been lost or taken so far in the current draft, and the next block (marked for the next season) is the actual order of the teams. I'll have to do more digging and see if I can improve the output.

[gottimd]

Originally Posted by gstelmack

Okay, looks like it is off by a year. I think the first block marks what picks have been lost or taken so far in the current draft, and the next block (marked for the next season) is the actual order of the teams. I'll have to do more digging and see if I can improve the output.

So the order never needed to manually altered? I thought it gave you a default (like alpha) and you had to resort using the drop downs.

Dammit, I want those 10 minutes back of my life I spent redoing the order!

[gstelmack]

Originally Posted by gottimd

So the order never needed to manually altered? I thought it gave you a default (like alpha) and you had to resort using the drop downs.

Dammit, I want those 10 minutes back of my life I spent redoing the order!

You are thinking of the Mock Draft feature in Analyzer, while he's asking about the DraftOrder.csv file exported by Interrogator.

But yeah, having Analyzer read in the Interrogator output is a possible future feature ;-)

[Subby]

Greg - I just wanted to show you an example of the stuff I can do for our league - and its only possible because of your amazing utility...thanks a ton.

[cuervo72]

Is there an easy way of determining coach role for the current year? I am trying to grab that for the current year, and staff.csv has contract, salary, team...no role (that I can see). We are in the offseason leading to 2010, so there are no entries in staffhistory.csv yet, so I can't grab it from there. Is there any way role could be included in staff.csv?

[gottimd]

Originally Posted by cuervo72

Is there an easy way of determining coach role for the current year? I am trying to grab that for the current year, and staff.csv has contract, salary, team...no role (that I can see). We are in the offseason leading to 2010, so there are no entries in staffhistory.csv yet, so I can't grab it from there. Is there any way role could be included in staff.csv?

On the tables, it shows you the value. See the Lookuptables.Csv output.

Value	StaffRole
0	Head Coach
1	Off Coord
2	Def Coord

You'll have to do vlookups or something on the Staffhistory.csv pages as well on the "Role" column, If I'm understanding your question correctly.

[cuervo72]

Yes yes, I understand the roles are in the lookuptables.csv. My problem is that the staff.csv file has the following entries:

staffindex
lastname
firstname
team
salary
qb
rb
...
age
wins
losses
ties
playoffs
bowls
contractyears
experience

So, I don't see a 'role' column in staff.csv.

Thinking logically, I go over to staffhistory.csv . Hey look, it has a column for role. That's great. So I scan for year...ok, 2003...2004...2005...2006...2007...2008...2009...UH OH, no 2010. Well, we're in 2010, only in the offseason.

So for instance - I just hired Perry Hopkins as my head coach. His ID is 21. Bingo, there he is in staff.csv. But there is no role there. So I look for him in staffhistory.csv. Wait, what's that? He's a rookie, and doesn't show up there?

THAT's my problem.

[gstelmack]

Originally Posted by cuervo72

Is there an easy way of determining coach role for the current year? I am trying to grab that for the current year, and staff.csv has contract, salary, team...no role (that I can see). We are in the offseason leading to 2010, so there are no entries in staffhistory.csv yet, so I can't grab it from there. Is there any way role could be included in staff.csv?

I dug a bit more and found the "current role" and "retiring this season" flags for the staff guys. I'll try and get a new release out soon (I'll try to dig into the draft issue as well and see if I can put together a better output for it).

[cuervo72]

Excellent news Greg - thanks!!

(interrogator is teh r0xx0rs )

[beatle]

Can you give directions on how to use this file? I just downloaded it and tried to use it with out success.

[Subby]

Originally Posted by beatle

Can you give directions on how to use this file? I just downloaded it and tried to use it with out success.

Here

Good html help for each individual program included in utility...

[cuervo72]

Is sacks being off by a factor of 10 in the playergame csv's (defenders) a known issue?

(I'm accounting for it now, but was wondering)

[gstelmack]

Originally Posted by cuervo72

Is sacks being off by a factor of 10 in the playergame csv's (defenders) a known issue?

(I'm accounting for it now, but was wondering)

No. I may have to check. Some of the fields have extra bits coded into them (for example, injuries can be +1000 for severity or career ending or something similar), so sacks may have something going on there as well. Most of Interrogator is just grabbing the data straight out of the binary.

The key is I'm so busy writing these utilities that the Viewer and Draft Analyzer are the only two I actually use and look at on a regular basis, so I rely heavily on all of you to tell me what's wrong with Interrogator

[gstelmack]

Changes include:

• Interrogator now has 2 new columns in the Staff.csv file:
  • Role: Current role on current team. If this is "3", the staff member is no longer active in the game.
  • Retiring: If this is "1", the staff member has retired this offseason and while visible in the game is not available to be hired.
• Interrogator has split off a new file from DraftOrder: DraftStatus.csv. This has information about any ongoing draft. DraftOrder.csv should now have the correct team's assigned for the next draft (or current if ongoing) and info about future drafts (traded picks are marked here).