AI difficulty sim test

[voodoo_magic]

EA PGA Tour,
In straight simming the AI difficulty is very disappointing. It's kind of hilariously similar to Madden where the sliders either do nothing or a one point difference leads to a lot of roughing the passer calls.

In the first test of the Korn Ferry tour first 6 tournaments I ran AI difficulties of 40 through 90 in increments of 10 and also did 95,99 and 100.
I compared the winning score, 10th place, how many were under par and what the 50th score was.

From AI setting 40-99 there were no increasingly better scores. Sometimes an AI of 40 had a lower score than 99.
Same thing with 10th place, under par scores and 50th place, AI difficulty had no effect.
I also did a test checking quick rounds and full, no difference.
Then I did a test of starting on the PGA tour with the above AI difficulties, quick rounds or full rounds and got the same type of results as the Korn Ferry where no effect of changing the AI difficulty.

Except for AI difficulty at 100.
In both Korn Ferry and PGA tour finally there is a drastic change with winning scores in the -22 to -25 instead of -14 to -18 and top 10's of -17 to -19 instead of -5 to -9

In our careers maybe it's programmed to put out these similar AI scores as a base and then react to how we are doing or to have worse scores if the AI is set to say 40 and have better scores if it's at 90 to challenge us?

I've only played full rounds in my career into the Q-school part so far so some of you may have played a lot more and into the PGA tour where the AI is posting increasingly better scores as you increase the AI difficulty because you are actually playing out the rounds.

Anyways, I thought it was interesting enough to post since we're all going to want to increase the AI difficulty to challenge us as we get better but it's concerning that it may not work properly.

[Peterswan62]

The ai on pc is totally broken. Its actually been ported straight over from Rory golf. And we all know how broken that AI was.
This is EA. The wont spend money on AI. And judging by the comments everywhere on the game, hardly anyone seems to care.
People think that if it looks good then the game is good.
The career mode is virtually unplayable at present.
The ai does not move scores forward after round 1 unless you are on 100 difficulty.
I played on 60,65,75,94,96 and 100. Apart from at 100, the scores stand still.
As expected from ea. They haven't done anything with the AI.

Oh yes, I if you have ai at 100 then the scores are ludicrous.
Another EA sports game thats style over content.

Realism is not in the visuals. Realism is in the gameplay.
Sadly most people dont seem to care about realism.
EA love these people

[Dawgfan1]

Good tests, I was wondering if the AI difficulty setting may have been broken because I saw TheApexHound put his AI setting to 60 or something but was getting similar scores as I saw when I test simmed a few tournaments. I do wonder if you have to play them yourself to get the higher scores, I think that's worth considering (shouldn't really work like that though). I might do some tests of my own.

Quote: Originally Posted by Peterswan62

The ai does not move scores forward after round 1 unless you are on 100 difficulty.

I would challenge you to sim through an actual PGA Tour event (not a major) and see what happens. I just finished my Q-School on 50 AI and the leader's score went from -3 after the first round to -8 overall. Didn't feel like they stayed the same to me.

Quote: Originally Posted by Peterswan62

Realism is not in the visuals. Realism is in the gameplay. Sadly most people dont seem to care about realism. EA love these people

Again, sim through an actual PGA Tour event. I've done it and got -23 on The American Express, the IRL winner is 23-27 under the last few years. Also got -13 on the Farmers Insurance Open, the IRL winner is 13-15 under the last few years. Maybe the AI settings slider is broken, but the base scores are undoubtedly realistic.

[sadler]

I think simming through gives you unrealistic results. Slider and scores seem to work better when you play through tournaments.

[Dawgfan1]

Alright, did some sim tests of my own. I went through and simmed 5 first seasons of 5 different careers at 0, 25, 50, 75, and 100 AI difficulty. Used a short season length to make simming faster. Here are the results:

For the 0 AI difficulty, I got really high scores as should be expected. Only one real-life tournament had a score lower than -10, the American Express at -13. Every other tournament hovered around -3 to -7.

For 25 AI difficulty, I saw a significant jump in scores to double digits under par for every tournament except the US Open. Most between -13 to -17, a couple in the 20s. This level of scoring remained for the 50 and 75 AI difficulty, with pretty similar tournament scores across all 3 difficulty settings.

For 100 AI difficulty, I saw a final big jump in scores. Almost every tournament was more than 20 under par, with 2 exceptions (Farmers Insurance Open and US Open). Winners generally ranged from -21 to -24.

Based on these results, it seems that the AI difficulty slider isn't completely broken, but it doesn't really change the scores except at the max and min setting. So you can make it easier or harder on yourself, but there's not really an inbetween. This should be better tuned.

However, I think there's a chance that this could be a sim-only thing and when you play the tournaments yourself then the AI scores will reflect the difficulty setting. I saw Respawn do the US Amateur with 70 AI difficulty and his highest AI score was -11. When I did the same tournament with 50 difficulty my highest AI was -4. This would probably need proper testing though. There's also the possibility that the game modifies CPU scores to a point to adjust to your player's scoring (i.e. if you score -30 the CPU scores increase to make your lead smaller). Respawn shot -32 on his US Amateur so this could be an explanation for his high AI scores.

Lastly, I looked up the 2022 scores of all IRL tournaments and compared them to my sim results at 50 AI difficulty to test their realism. I've noticed that the winning scores in this game can vary by up to 4-5 strokes in different sims, so I considered anything within 5 strokes of the IRL score to be realistic. I found that for all but 2 IRL tournaments, the sim score was within 5 strokes of the 2022 score. For the 2 that weren't (Arnold Palmer and Wells Fargo), the 2022 score was abormally low and choosing a more median score in recent history put both those tournaments within the 5 strokes to the sim results. I think that shows that the default sim results in this game are realistic.

Here's a link to my scores: https://pasteboard.co/Y17YEfl8PszH.png

TL;DR: The AI difficulty setting seems to only change simmed winning scores from default at or near either extreme (0 and 100) and should be tuned by EA to provide inbetweens, though playing the tournaments yourself may provide scores more aligned to the AI difficulty setting you've chosen. The default 50 AI difficulty simulation results are generally realistic.

[Zinger]

Interesting results Dawgfan1. Do you (or anyone else) know if the AI slider can be adjusted from round to round or only at the start of a tournament? I haven't started my career yet but would like to be able to adjust the slider from round to round to allow for changing course difficulty (eg calm wind to strong wind). In my experience the AI scoring in golf games does not automatically adjust for this sort of thing.

[Oblong]

Good stuff.

Been having conversations with devs about widening the gaps of scores using the AI tuner. It is odd that this one has a 0-100 slider when it seems possible there may only be a range of 3 or 4 adjustments.

As someone with only a minor in statistics I can at the very least envision some simply ways they could implement something that offers good random probabilities that also can allow for a few outliers to give some interesting varieties of outcomes.

[Dawgfan1]

Quote: Originally Posted by Zinger

Interesting results Dawgfan1. Do you (or anyone else) know if the AI slider can be adjusted from round to round or only at the start of a tournament? I haven't started my career yet but would like to be able to adjust the slider from round to round to allow for changing course difficulty (eg calm wind to strong wind). In my experience the AI scoring in golf games does not automatically adjust for this sort of thing.

I haven’t tried changing it round to round but I guess it only affects the current tournament.

The last golf game I played, TW14, did a pretty good job adjusting AI scores based on weather conditions (worse scores in windy conditions, etc.) but I don’t have much experience playing in high winds in this game to know if or how AI scores are affected.