Videos
I have an office chair I like that I went through a lot of effort to find. (It's similar to another chair I had used before for almost 20 years) I've replaced the base before because the old one's wheels snapped. (After around 5-6 years) Two years later, this new base (B09K67YQW4 on Amazon) broke now too in the same way. I'm incredibly frustrated after a very stressful week in a very stressful point in my life. I'm just looking for a base replacement for an office chair that is built to last. It was huge struggle to get the old base off for the first replacement and having to do it again is already driving me up the wall in insanity.
Best option - drill hole out to 3/4", cover a 3/4" wooden dowel with wood glue, stuff it in the hole, wait for glue to dry, cut it off flush, drill new hole.
Wood filler is not a quality repair you should expect to last.
You might have enough room to drill your small hole beyond the current hole.
For sure wood filler isn't going to work... if the previous suggestion (which is quite good) won't work for you, for whatever reason, perhaps something like Bondo would. You could put the new caster stems into small baggies, or finges cut from rubber gloves to seal them off from the epoxy. I've done this in very similar situations and it works fine to prevent the epoxy from getting into the new parts.
Having said that, the idea of gluing dowels into the holes and re-drilling is the classic solution.