Videos
I am wondering if buying a 2013 Mac Pro is still worth it today
Only if it is extremely low cost and you don't even have a computer to start with.
the 2013 Mac Pro can undergo CPU upgrades
...
is there major hidden costs to upgrading a older Mac Pro?
You should know that the 2013 Mac Pro can only upgrade to specific CPU models. The 2013 model uses socket 2011, and Ivy Bridge (3rd gen) was the last generation to use that socket. The Xeon E5-2697V2 is the fastest CPU that Mac supports in that socket. That CPU is old, discontinued, has security flaws, and not nearly as fast as what you can buy now.
would the CPU upgrade to say a current gen i9 be feasible for performance of large computations?
As explained above, the i9 CPUs are much newer and a totally different socket. It's not even possible to put one in a 2013 Mac Pro.
if I were looking to use it you would a regular PC where you can continuously upgrade is that feasible?
This is hard to answer if you're really set on using a Mac OS.
Building your own PC is the best choice for upgrading it later because everything can be replaced over time like the Ship of Theseus.
However, building a custom PC almost always forces you to run Windows or Linux because the non-Apple hardware is unsupported by Apple. They make it very hard to install Mac on hardware they don't make and there are a lot of components you have to avoid to ensure it will even work.
So if you really want Mac, your best plan for upgrading is to save until you can afford something newer than that 2013 model. And do not buy any Apple computer with the expectation of upgrading the CPU these days.
It depends what you need it for.
I still have several older Mac Pros, up to 2012 spec [cheesegraters], because they're still great for running audio DAWs. The high clock speed & 24 core HT performance makes them still very useful, so long as you don't mind not being up to date on either OS or software. A Metal 2-capable GPU will get them up to Mojave, which is very stable. [You can force them up to Monterey, though I've never considered it worth it].
The other advantage of the old cheesegraters is you can get 6 drives in them before you even start modding. Once you add PCI SSD adapters, that climbs still further. I've 2TB SSD & 14 TB HD in mine [you can use HD drives larger than 4TB, but you need to buy modified drive sleds to mount them, because someone decided to move the screw holes on bigger drives.]
You can put any PC GPU in them if you want - but if you want full macOS support, use ATI/AMD & buy specifically Mac-flashed. There is no official support for NVidia after High Serra.
You can get a full-spec 3.46GHz* 64GB RAM cheesegrater for under $£€ 400 [check whether it says Mojave-capable & exactly what GPU is in it]. The 'trash can' 2013 still fetches closer to 500, is only fractionally faster [or even slower, depending on which model], and you can't fill it full of drives.
*The 3.46GHz processor was never available from Apple when these were new, so it's already been modded by someone. You may find under the hood that it is either a 2009 4,1 that's been flashed, or a 'real' 2012 5,1. There is no practical difference to you as an end user once the mod has been successfully completed. The machines were virtually identical when new, but used different CPU series [Nehalem vs Westmere.] Once the mod is done, you have Westmeres.
I’m old. I go back to the time when the “Trash Can” (also The Cube, for that matter) was highly coveted and totally unaffordable. Now 2013 Mac Pros are practically being given away and nostalgia combined with the prospect of an upgrade from my current system (2012 Mac Mini with 16gb RAM and 1tb SSD) has me window shopping. I work with music, sound design, software synthesis and Python (I compose with Python as well as use it for building utilities on the fly). Thoughts? Warnings?