I'm not really sure I can help you.
It also happens to me when I use Edge. I think in my case it's because Windows/Edge decides to put the tabs I'm not currently using into a sleep mode and, if they stay that way for long, once you go back to them Windows/Edge has to wake them up, taking a couple seconds while checking all the drives for caches or other info it may need.
I haven't tried it as it doesn't bother me enough for the hassle but, if my situation is similar to yours and you want to do something about it, there's a way to stop Edge from putting tabs to sleep (Settings => System and performance => Optimize performance section => turn sleeping tabs off). You can also create a whitelist in the "Never put these sites to sleep" option and adding the URLs of those sites, so they never go to sleep while the rest do, freeing resources in case you need them.
Please excuse my bad English.
Former gaming PC: i5-4670k@stock (for now), 16Gb RAM 1600 MHz and a GTX 1070
Current gaming PC: R5-7600, 32GB RAM 6000MT/s (CL30) and a RX 9060XT 16GB
Steam / Live / NNID : jonxiquet Add me if you want, but I'm a single player gamer.
