FOF2001: Shotgun Predictability

[Morgado]

One thing that's bothered me recently as I try to figure out how to run a Run and Shoot offense from within the FOF engine is that shotgun formation pass plays seem to be extremely easy for the defense to familiarize themselves with. I'm guessing that "the defense seemed very familiar with that play" shifts the success distribution of the random variable for play outcome.

The thing that gets me is that I almost never see that message except when shotgun formation plays are called. And it isn't only when the computer is running ridiculous desperation plays like Shotgun 5 WR or 4 WR sets, but also some 3 WR sets. If the defense seemed familiar with other formations more often this wouldn't bother me as much.

When your computer controlled coach ever uses a shotgun formation, it's almost always a pass. I mean it's like they don't even bother trying to keep the defense honest... maybe running twice or three times the entire game out of a shotgun set. Usually I see these ridiculous playcalls like Shotgun 4 WR vs. Dime, 4 Deep, Pass Aggressive. Gee I wonder why the pass wasn't completed.

Lately, I have been trying to experiment with very low shotgun percentages and throwing out of neutral and even run oriented formations. Seems promising but I haven't simmed too many season with such offenses. I think this is what i'll be looking at for the next couple days at least. Shotgun seems to be ultra-predictable or something... almost to the point of not being worth using at all.

[dolfin]

I have actually had "somewhat" of success using shotgun. My last season I accumalated 2211 yds out of the shotgun over the season. 500 yds more than the next closest formation (1754 single back). But, the completion percentage was much lower, 39% compared to 50% out of the single back formation. I have been to the playoffs for the last nine years in a row. Maybe I would win more Bowls if I backed off the shotgun a little.

[Morgado]

Originally posted by dolfin:
My last season I accumalated 2211 yds out of the shotgun over the season. 500 yds more than the next closest formation (1754 single back). But, the completion percentage was much lower, 39% compared to 50% out of the single back formation.

Yeah, this jives with what i'm seeing during scoreboard simulation. Especially after my high percentage pass offense experiment, i'm beginning to think that formation may link to the types of routes called. So if you go shotgun, the coach will likely call longer routes with lower percentage completion but gobs o yards any time it is successful. The more conservative formations like the Strong/Weak and Split will run typically shorter routes that are higher percentage but rely heavily on run after the catch for significant gains. Single back would be in here too.

The other thing that may tie in with the formations are receiver sets. I have a feeling that there are tables and adjustments that skew where the extremely predictable 4 WR and 5 WR sets are employed... this may be even more of a "legal" tipoff to the CPU opponent as to when you're going to throw.

At this point, i'm not sure exactly what to make of Jim's note in the help file on which formations are oriented to what plays means. Like, what exactly does Single Back being slightly oriented toward passing mean? Does this indicate a higher mean completion rate? A higher mean yards per completion? A narrower standard deviation on completions? Etc... right now, just hunches.

[JW]

I've had great success running a pass-oriented offense using three formations, the split backs, one back, and shotgun, simply varying the percentages for each in my game plan depending on the situation. Generally in passing situations I run 50% shotgun, 40% one back, 10% split backs, or something near that. I seem to be less productive when I put any one formation at too high a percentage, anything over 60%. I sim all my games, by the way. In spring training, that is all I concentrate on. This isn't exactly run-and-shoot, but it has worked well for me. I have also used these formations with more balanced attacks.

[Morgado]

Originally posted by JW:
In spring training, that is all I concentrate on.

Okay, what kinds of expertise ratings are you coming out with and what's the cohesion for the passing game and rushing game (offensive line) units? I'm curious because I haven't really tried doing the all-out pass camp much. Pretty much just with empty cupboard squads and I think the atrocious cohesion/expertise results from camp are what was killing my results.

Intuitively, I felt a three formation offense that goes something like your 10/40/50 mix should work well (reversed in running situations). Generally, what house rules are you imposing and what quality receivers, linemen, and backs are you using with this setup?

BTW yeah, I don't think a "real" Run and Shoot is possible in FOF since the game runs off of stat tables with no real individual plays and option route running. True Mouse Davis/June Jones Run and Shoot would be something like 80% Shotgun, 20% Single Back except the back would be a FB and not an RB. Set balances would be 0 on anything using a TE, 0 on Goal Line, 100 on FB as Primary Ballcarrier, and like 100s on both 4 and 5 WR sets.

Of course, with something this damn predictable, we can imagine how badly this thing would get thrashed in FOF's engine... heheh... What you're using looks pretty close to what I was thinking with some funky pass distance balance.

[JW]

All right. Here are some numbers. First, a disclaimer. I don't claim to be an expert on strategy. The primary thing I do is try to get the best players, particularly on the line and at QB. Then two really good WRs, then a good TE, then good RB, then good FB, in that order. I try to get a QB with balanced ratings and a couple of WRs who catch a lot of passes.

But I read some time ago in the old FOF2 forum that several people had a lot of success just concentrating on 3 formations. That seemed to be enough to get enough variety to confuse the AI and allowed you to really bump up the expertise. I've used various 3-formation combinations and that has seemed to work for me.

I happen to have latched on to a really good RB, so I've actually gone back to I-formation (instead of split), one back, and shotgun. I think that helps my running game. And I'm running a more balanced offense right now. Here are my current percentages of playcalling for I, one back, and shotgun, per situation:
Run: 45-45-10
Likely run: 70-30-0
Pass: 15-50-35
Likely pass: 0-50-50

Here are my passing percentages by type:
13-50-25-8-4

Note that I also really bump up my 2WR and 3WR ratings, to 70, and really bump down my others.

Also, you have to look at your run percentages in situations. I adjust that, particularly the % run on 1st and ten, depending on my team. I also adjust it once or twice during the year to confuse the other teams, and I adjust it per game in the playoffs based on the opponent I'm playing. I've run it all the way from 30 to 75, depending on my team and the opponent.

My current expertise is I-70, One back 70, shotgun-70. Note I also put a lot of emphasis on defense, particularly LB, since it seems easier to find good LBs. I load up there.

As for cohesion, my passing is currently 64 and OL 81.

These type combinations have given me what is usually the top offense in the NFL.

But I have to wonder if it is based more on player quality than strategy.

Finally, I look for a high-rated QB who is a winner. If my team struggles, I get a new QB, until I find one who wins games.

Hope this helps some.

[Morgado]

Originally posted by JW:
I don't claim to be an expert on strategy.

No problem. I don't think any of us have pretensions to being an expert or guru of any kind.

But I read some time ago in the old FOF2 forum that several people had a lot of success just concentrating on 3 formations. That seemed to be enough to get enough variety to confuse the AI and allowed you to really bump up the expertise. I've used various 3-formation combinations and that has seemed to work for me.

Hmmmm. Okay, mostly i've been working a four formation package and haven't tried going to three. I think i'll try that out.

Here are my passing percentages by type:
13-50-25-8-4

Note that I also really bump up my 2WR and 3WR ratings, to 70, and really bump down my others.

Okay now this is interesting. This is pretty close to what I originally played with as a standard offense. About a 15-55-20-6-4 mix on passing distances. Are you changing the percentages or is this what the computer scout is defaulting to postcamp? I used to always have to bump up the midrange passes from like 16 to 25. The 2WR and 3WR increases makes sense since you're throwing to your WRs a lot. I wonder if calling pass plays to your TE reduces blocking effectiveness. Hmm, gotta figure out some way to test that - might be a good argument for more WR directed throws.

I also adjust it once or twice during the year to confuse the other teams, and I adjust it per game in the playoffs based on the opponent I'm playing. I've run it all the way from 30 to 75, depending on my team and the opponent.

Okay this is one thing i've been laying off of, and have been leaving my run percentages untouched the whole way. I think this might be the reason i've been forced to four rather than three formations to get enough variety.

But I have to wonder if it is based more on player quality than strategy.

To be honest, I think the real key to the offense's success is the changing of the run percentages midseason. I remember someone mentioning that the computer teams take a few weeks to "catch up" with your offensive scheme after you change tendencies in FOF2.

My guess is that the computer takes the run percentages for your actual playcalls over the past three or four games, does a rough average, and uses that as a baseline expectation for how to defend you. If you change tendencies say every four weeks, that really will throw some red herrings in front of their scouts. Usually what happens as I play a season without changing percentages at all is that the first couple games I absolutely manhandle everyone. Then 5 to 6 weeks in, my offensive production drops off drastically as everyone knows how to play against me. I think you should try looking at how changing or leaving the run percentages changes your offensive output - I know i'll be taking a look at that effect. Maybe sim eight games and then reload and sim four change then sim the last four to compare. My guess is the no-change straight eight will not be as successful as the with-change eight... particularly on the final four games.

[B}Hope this helps some.[/b]

Yep! Gives me a lot of things to think about and test out. Thanks!

[JW]

As for the passing percentages by type, I always have to bump up the midrange percentages to achieve what I want.

As for changing my run on 1st and ten %, I dont' get too analytical with the game, but I do notice that I start strong and then lag a little, and then a after I change my percentage I usually get a good positive bump.

In the playoffs I often get rather radical. That is where I go down to 30% or up to 80%. I might also throw a lot more long passes if the opponent uses a lot of bump and run and blitzes. That seems to work well. I also really load my defense in the playoffs against the opponent's offensive tendencies. I don't worry about that during the season.

[Morgado]

I have been getting some pretty interesting results from playing with run percentages and throwing distances. The two quarterbacks who get most of the snaps (starter and backup because starter got hurt for like 1/3 of a season) are very low rating guys - players you can get in the second and third rounds. Average everywhere except both guys are actually not bad at medium and deep passes with some third down and short/screen ability. Throw power is abysmal on both but accuracy is decent.

Anyways, I tried taking baseline league average run percentages and adjusted them so I would throw on 1st and short, 1st and mid, as well as 2nd and short and 3rd and short. Pumped up the run for 1st and 10, 2nd and 10, 2nd and mid, and 3rd and mid.

Then I went to pass distances and made some adjustments to fit my quarterbacks: 10/40/40/5/5

Looks pretty good. Even with a vanilla receiver corps and a non-all world offensive line, I got good production out of the offense - especially with the running game. Granted, I did draft a very good running back, but he isn't maxed out and has crapola scores on carrying (like 30) and endurance (like 60). still gets about 1250 yards in 15 games worth of carries (usually misses one to injury).

All this, and I didn't have to change my run percentages at all during an entire season and still went 11-5. Not sure if it was a fluke, but even in an 8-8 season got good offensive numbers without really touching the run percentages.

[JW]

Merry Christmas. Someone else wrote about changing the % to throw more on 2nd and short or 1st and short, etc., and I have tried that. It is hard to tell if it worked well, but it seemed to. I don't really adjust my 1st and ten run % as much as I may have implied. Mainly I set it at the start of the season and only change it if I think I need to, maybe midseason, or after a QB or RB injury, or something like that. I sometimes just leave it. But I do adjust it for every playoff game, after reading the scouting report on the opponent.

[Karlifornia]

Man, if my teams can't run the ball, it seems like we're destined to suck. There aren't many real life teams that can maintain a high level of success without a running game, so I guess I can't complain. I'll be on the lookout for that "familiar" phrase.

[OldSchool]

JW -
I thow on 2nd and short almost all the time (I call my own plays, but if I was setting % I'd put it up around 75 i think).

It definitely works. A lot of teams do it IRL, too, so I don't think its cheating. (Its certainly no lock, and I've been burned plenty of times going deep on 3rd-1 and 4th-1) Just sort of seems to flow w/ the game plan

[JW]

I have read about the throwing on 2nd and short elsewhere, possibly from you, and I do use it sometimes. But since I only sim my games, and I don't do a lot of game analysis, I'm not sure how well it works for me. But it certainly makes sense against the computer.

[Subby]

To be honest, I think the real key to the offense's success is the changing of the run percentages midseason.

Sorry to get in on this one so late...

Something I have been trying (with success) recently is to add 20 to my first and ten and second and long run percentages, while subtracting 20 from my second and short and third and short run percentages. I play that way for three games, then switch or flip-flop. The advantage is that I can keep the defense guessing, while maintaining a balanced offense.

My pass percentages are as follows:

15-50-30-5-0

Which I might tweak a bit, but the thinking is that my speedy receivers are going to pull a Jerry Rice and get lots of yards after the catch. I change these percentages less - depending on the primary pass defense played by the opponent - but it seems like most teams like to play zones, making this a highly efficient, west coast type of offense.

As far as Formations are concerned, I zero out three: 0 WR, 4 WR, 5 WR. For starters, few teams are going to have more than three decent wideouts - plus 4 and 5 wideout sets are conducive to picks. The 0 WR set is just too...boring

I run 4 different formations and spread distribution out evenly (25-25-25-25). This seems to keep the defense from keying on any one formation...

Good stuff here. I always enjoy reading your stuff, Morgado.

[Morgado]

Subby,

Thanks for the kind words. I try to make my posts worth reading. I totally agree with your WCO sentiment on short passing. My own passing setup 10/40/40/5/5 is very similar to your ideas and lets the wideouts make plays. The thing I like best about making huge short and intermediate range percentages is that it really makes the backs and TE more involved in the offense. And like I post all over the place, I just love FB and TE based offense.

Just as a reaffirmation of my faith in yards per catch, I noticed while watching bowl games in Vegas that indeed most of the yardage comes after the catch in today's offenses - even college ones. The most obvious example of relying in isolation plays to playmakers is the now-prevalent WR screen. Lots of times we see slants and hitch routes being taken from a 2 or 3 yard hot route into a nice 8 yard completion.

Morgado Sports Network went 9-1 Against the Spread. It was a very nice trip.

Anyways, yeah I should probably jack down the 0 WR set a little more myself... I tend to almost never zero out anything except for the front i'm not playing on defense due to the problems we've seen zeroes cause (willingness to play injured etc). Still working on that Base Nickel defense and have it jacked up to 70 permanently with a 65 blitz percentage. I hope Jim puts Nickel as a base personnel option in TCY since so many teams (Nebraska even!) play it as a base set.

[JW]

Last night I tried putting my 0 WR and 5 WR on zero, my 1 and 4 WR on 30 and my 2 and 3 WR on 80. My QB is a very good one, but his efficiency jumped up to 103.7 for the year (12 points above his career average), one of my all-time great ratings in a 30-year history. My Saints won the Super Bowl, for the 6th time in 7 years after letting the Jets win it "last year." So I can't say for sure, but this seems to work well. It certainly didn't hurt.

On a side note, my QB remains disgruntled and has been his entire career, even though he now has a long-term contract paying about $25,000,000 per year. He keeps resigning, and he puts up great numbers, but he sure is a pain in the rear.

[Fritz]

I have tried a no-tendency approach to gameplanning.

I frequently set my gameplan to exclude shotgun and single back (If I have a FB worth looking at) and skew the game to go with 1,2 or 3 WRs (I am not sure how to get 5 WR out of the I.)

I will then mask my run/pass tendencies by setting each formation to about the same % (For instance, I will be in Split 25% of the time, and I am equally likely to pass or run from this formation.

I will further mask the run by setting my preferences to something like 10,15,15,20,15,15,10.

With a balanced attack I get very good results.



------------------
I look at spelling as a creative art

[OldGiants]

>To be honest, I think the real key to the offense's success is the changing of the run percentages midseason. I remember someone mentioning that the computer teams take a few weeks to "catch up" with your offensive scheme after you change tendencies in FOF2.
<

This is one topic we beat up considerably without resolution last year.

Are you talking play-calling mode, or simming?

Because if you're talking simming, how can the defense "adjust" to your play-calling, when it goes by the standard table? That is, I've found no evidence whatsoever that there is an adjustment algorithmn in the simming routine.

We all occainsionally get improved results from changing our game plans in mid-season, but often we don't.

Other than "somebody mentioning it" I've never seen any credible evidence that there is anything beyond the defensive table in simming game plans.

[Morgado]

Originally posted by OldGiants:
Are you talking play-calling mode, or simming?

Because if you're talking simming, how can the defense "adjust" to your play-calling, when it goes by the standard table? That is, I've found no evidence whatsoever that there is an adjustment algorithmn in the simming routine.

I'm tlaking simming here with no player control. I'm guessing:

1. The CPU goes into the game expecting your team to play like it has over the past say 5 games as a weighted lag. It doesn't know what your gameplan percentages are set to, but it can take what your team actually ran and try to run a maximum likelihood on that. When it estimates these MLE values, it could use these values as the "base" plan for the game.

2. The CPU will play the game and at specified intervals will update its MLE estimates based on what you are actually doing during the game. This would inviolve calculating several likelihood functions, but ach of them would be extremely simple. So computation would be very, very fast on anything more powerful than a P5-60. So fast, I would argue we would not notice it even in fastest simming speed.

3. The CPU continues to adjust both offensively and defensively to what new information it is being presented with via Bayesian update and MLE recomputations. So the CPU coaches for both the opponent and your team are behaving as rational economic agents.

[OldGiants]

I understand everything you've written, Morgado, this is what happens in many other football games. But you aren't addressing my point:

What are the defensive tables used for? They _never_ change. These tables are used for all offensive/defensive selections by the CPU.

We've all written about the second and short pass offense. Well that would not work if your intelligent AI theory were true. After a few games, the CPU would catch on. But it never catches on to the second and short and pass because it is locked in to looking for the run 75% of the time.

It has to be one way or the other. Either the AI uses the tables all the time, or it adjusts and the tables are irrelevant. The evidence from the second and short pass offense (and the 75% run offenses, too) is substantially on the side of the CPU never adjusting.

Also, I'm pretty certain that Jim G admitted on the old MB that the tables are always used and there is no AI adjustment for simming games. I wish I'd kept that post, but its' lost to eternity.

If you can explain the defensive tables and the adjustment process, I'm willing to listen.

[Morgado]

Below is a different view of gameplan usage as what I talked about above... OldGiants' comments provoked some thought and modifications to my guesses.

Originally posted by OldGiants:
We've all written about the second and short pass offense. Well that would not work if your intelligent AI theory were true. After a few games, the CPU would catch on. But it never catches on to the second and short and pass because it is locked in to looking for the run 75% of the time.

My guess is this. The CPU coaches take our table gamplans and use them as a base template to work off of when planning to call a game. This is the defensive percentages used before a single down is played during the game. Now, as the game goes on, the CPU coaches see what is actually called. They use this to modify the percentages *in game*. I'm not talking about changing percentages from game to game - i'm talking about changing percentages as the thing actually runs.

It has to be one way or the other. Either the AI uses the tables all the time, or it adjusts and the tables are irrelevant. The evidence from the second and short pass offense (and the 75% run offenses, too) is substantially on the side of the CPU never adjusting.

The way i'm looking at how the CPU uses the table sort of solves this, but it is unproven. I haven't ever watched whether the defensive playcalling on second and short really evolves over the course of the game. My hunch is that if a CPU coach saw enough 2nd and shorts, it would be able to update and adjust to the tendency. So you wouldn't see the CPU adjusting in say the first half. But by the time you hit the mid third quarter, the defense should have adjusted. Part of this evolving AI is reflected in the fact that later in the game, the defenses "seem very familiar" with more plays.

Also, I'm pretty certain that Jim G admitted on the old MB that the tables are always used and there is no AI adjustment for simming games. I wish I'd kept that post, but its' lost to eternity.

Okay, if you guys have gotten that from Jim, then I guess there's really nothing to speculate on. Judging from that, it would appear the tables are used hard and fast and the 2nd and Short is a way to trick the AI.

The 64 dollar question at that point then, is if the CPU controlled teams change their percentages between games at any point in the season. If so, what are they judging against? Baseline league average tendencies? Average tendencies of remaining opponents? Etc... Suppose Jim's comment were true for FOF2, but this in-game adjustment is one of the "under the hood" improvements to FOF... heh I donno. All of this is speculation.

Another interesting idea that came to mind while doing the two back 75 experiment was the difference between short yardage and breakaway speed ratings. Maybe we're looking at the RB ratings in totally the wrong way. What if Breakaway Speed has nothing to do with shifting bells and Short Yardage has nothing to do with mean shifting? What if it's as simple as:

When you run to a T or E hole, you use Breakaway Speed as the RB's "Run Skill" value.

When you run to a G or C hole, you use Short Yardage as the RB's "Run Skill" value. FB has no Breakawy Speed because they ain't supposed to run outside the box.

Bah, more idle speculation from me... heh. I shall now go back to trying to solve the problem of the 3dfx Difficulty Level setting on FOF2001 (DirectX 8 on Win98 didn't help).


[This message has been edited by Morgado (edited 01-16-2001).]

[ez]

Originally posted by OldGiants:
Also, I'm pretty certain that Jim G admitted on the old MB that the tables are always used and there is no AI adjustment for simming games. I wish I'd kept that post, but its' lost to eternity.

here's a link to the old forum: http://216.15.145.145/sideline/Ultim...assCookie=true

i don't remember such a comment from Jim, and didn't look for it, but maybe you can find it.

i do remember you and i discussing this a bit and getting nowhere.

(fact) every team starts with a "average" defensive game plan by default.
(question) is that modified before each game based on the opponent's particular offensive game plan?
(faint memory of a comment from Jim) "playcalling" amounts to the ability to detect trends.
(educated hunch) during a game, a coach can detect playcalling trends and modify his expectations and calls on defense accordingly.
(guess) "playcalling" determines the "length" of a coach's memory. i.e., EXC coaches remember many more plays than POOR coaches.

but what about an EXC rating for offensive playcalling? can the coach remember which plays worked better against particular defenses (an audible of sorts)? does he call plays from a "larger" playbook? is he more likely to call unexpected (but viable) plays?

i hope it's not as simple as an OFF-playcalling vs. DEF-playcalling mod at some point in the play-resolution sequence...

[This message has been edited by ez (edited 01-17-2001).]

[Kevin]

The only real evidence I've observed that the CPU teams may change gameplans is from the scouting reports on divisional opponents.

My scout's view of offensive tendancies seems to very rarely changes between the first and second game of the season. However, the scouting report of the opponent's defense often changes if there is at least a six week gap between games against that opponent. The noted changes usually revolve around the frequency of blitzing, and occasionally changing between a 4 deep zone and bump and run.

What all this means is still a guess for me.

[Ctown-Fan]

Currently setting up an experiment to test the theory about when defensive gameplans are "adjusted" by the AI. Hopefully I will be able to expand on this as my season contiunes.
Important Notes:
Coach has a Good reputation for offensive playcalling (the middle score which should not tip the scales in either direction)

Game Plan:
I am running three variations of Morgado's 75run offense
(a)50-50-75-default
50-75-50-default
75-25-default-default
(b)50-50-50-default
50-50-50-default
50-25-default-default
(c)50-50-35-default
50-35-50-default
35-25-default-default
the assumption is that with this sort of game plan the opposing scout will never be able to pin-down my offensive game plan before the start of the game. If the opposing coach is able to adjust the defensive plan throughout the game, then familiarity with the play will be seen in the game log.

play distribution
20-20-20-20-20-0
20-20-20-20-20-0
17-17-17-17-16-16
17-17-17-17-16-16

3te set to 0
5 wr set to 0
1-4 wr set to 60]

The process involves non-stop simming of the game until the end of the third quarter. At which point, I use the have coach call play button to slow down the action.

Since I am getting ready for bed the only update I can give right now is this. 3 games played and not one "defense looks familiar with that play."

Suggestions on how to make this a better test? Other thoughts on the process?

------------------
UMaine up one to #14!

[Morgado]

Originally posted by Ctown-Fan:
...then familiarity with the play will be seen in the game log.

I didn't think this showed up in the game log. I'll have to check that out... but it might be why nobody has been familiar with anything.

[Ctown-Fan]

I actually, was watching the game screen when I did that. I do think that you are right about it not being in the game log, though.

[OldGiants]

I simply wanted to continue this discussion since I've been remiss in not following this thread.

ez, yes, you and I have had many discussions that went nowhere. But they contributed to the pure research of the universe, so I'm happy.

I still think there is a major difference between play-calling mode and simulation mode. specifically, in play-calling mode the AI can adjust and make the "this seems familiar comment".

In simming, no one ever sees the comment, so does it exist? Again, I think the tables are all there is.

If memory serves, and it does so less and less with the mega-deaths of my neurons each and every day, there were several long discussions of the defensive tables in the game and the inability of the AI to adjust. Jim G's comment was an indirect admission that the tables were it. But FOF2000 could be different, of course.