Intermittent loss of internet

I have 1Gig fiber internet, but I'm experiencing intermittent internet connection drops once or twice a day. The issue eventually resolves itself after some time, but I've noticed that disconnecting and reconnecting the Ethernet cable to my switch or power cycling the switch immediately fixes it, although sometimes only briefly. Initially, I suspected a faulty switch, even though my Netgear JGS524 switch is relatively new. To troubleshoot, I purchased a new switch (TL-SG1024S), but I'm still encountering the same problem, ruling out the switch as the sole cause.

I've had AT&T replace my gateway (BGW320-505), but this didn't resolve the issue either.

This problem started in my newly constructed house where I had Ethernet wiring installed with 24 drops (2 in each bedroom, 4 in the living room, and 12 in my home office). I tested the drops after installation and found two that were faulty, but the rest were fine. All the drops terminate at individual wall plates in a closet in my office (although I originally requested the electrician to terminate them to a patch panel, this detail got overlooked).

I'm puzzled about what could be causing these internet access drops and why power cycling the switch consistently resolves the issue.

One point to add here... when I lose internet access, I can ping a remote site and although that fails, I can see that I get to the 2nd hop... Greenville SC in my case.  Weird.