I think things are much clearer if you differentiate between a vector (an object) and its coordinates (its representation).

When we talk about a vector, say in , we mean a specific point in it. When we talk intuitively about the coordinates of a vector, this is just actually just shorthand for its "parts" with respect to other vectors (the basis) which we have fixed as "building blocks."

For example, with the usual , the standard basis is and $\begin{bmatrix} 0\\1 \end{bmatrix}$. Then, a vector, say $\begin{bmatrix} 5\\6 \end{bmatrix}$ is actually $$\begin{bmatrix} 5\\6 \end{bmatrix}=5\begin{bmatrix} 1\\0 \end{bmatrix}+6\begin{bmatrix} 0\\1 \end{bmatrix}.$$ In this case, the vector is no different to its coordinates.

When we change coordinates, we change our reference point (our building blocks; our basis) but not the vector itself. For example, if instead I use the vectors and $\begin{bmatrix} 1\\1 \end{bmatrix}$ as basis, the vector $\begin{bmatrix} 5\\6 \end{bmatrix}$ is now represented by the coordinates $\begin{bmatrix}-1 \\6 \end{bmatrix}$ since $$\begin{bmatrix}5 \\6 \end{bmatrix} = -1\begin{bmatrix} 1\\0 \end{bmatrix} + 6\begin{bmatrix}1 \\1 \end{bmatrix}.$$

The coordinates tell us how much of each building block we need as these are the components that make up our vector.

If you wanted a slightly more involved explanation: When working with linear spaces, these transformations of coordinates are typically handled by change of basis matrices . These represent the identity map with respect to different bases and . What these do is that they take the coordinates of a vector in and then output the coordinates of with respect to .

In symbols, with ,

As an example, let be the standard basis, $D =\lbrace \begin{bmatrix} 1\\0 \end{bmatrix},\begin{bmatrix}1 \\1 \end{bmatrix} \rbrace$ and $v=\begin{bmatrix} 5\\6 \end{bmatrix}.$
We then have .

The coordinates of $\begin{bmatrix} 5\\6 \end{bmatrix}$ with respect to are then given by $$[\begin{bmatrix} 5\\6 \end{bmatrix}]_D = P[v]_B=\begin{bmatrix} 1&-1\\0&1 \end{bmatrix}\begin{bmatrix} 5\\6 \end{bmatrix}=\begin{bmatrix} -1\\6 \end{bmatrix}$$ as we have obtained prior.

If you want to visualize these, think of rotating, changing the angle between or stretching the and axis in and then find the coordinates of a vector with respect to the squished and rotated coordinates. You can also easily add translations, although these are no longer linear in the strict sense (linear operations fix the origin). If you involve translations, the transformations you are looking at for changes of coordinates will be called "affine".

For more basic information, I suggest the introductory book by Jim Heffron on linear algebra, which is readily available for free online: https://hefferon.net/linearalgebra/. It is a friendly read and has many examples and exercises.

This question is somewhat related to this question, with the main difference that you also have an translation.

Lets first make sure that the notation is clear:

A vector $\vec{v}$ can be represented in the initial Cartesian coordinate system as,

$$ \vec{v} = \begin{bmatrix} \vec{e}_x \\ \vec{e}_y \\ \vec{e}_z \end{bmatrix} \cdot \begin{bmatrix} v_x \\ v_y \\ v_z \end{bmatrix} = v_x \vec{e}_x + v_y \vec{e}_y + v_z \vec{e}_z, $$

where $\vec{e}_x$, $\vec{e}_y$ and $\vec{e}_z$ are the orthonormal basis vectors of the initial Cartesian coordinate system.

Similar, the vector $\vec{v}$ can also be represented in the new Cartesian coordinate system as,

$$ \vec{v} = \begin{bmatrix} \vec{e}_u \\ \vec{e}_v \\ \vec{e}_w \end{bmatrix} \cdot \begin{bmatrix} v_u \\ v_v \\ v_w \end{bmatrix} + \vec{o} = (v_u + o_u) \vec{e}_u + (v_v + o_v) \vec{e}_v + (v_w + o_w) \vec{e}_w, $$

where $\vec{o}$ is the vector connecting the initial and new origin and $\vec{e}_u$, $\vec{e}_v$ and $\vec{e}_w$ are the orthonormal basis vectors of the new Cartesian coordinate system, however these can be written as a linear combination of the basis vectors of the initial Cartesian coordinate system,

$$ \vec{e}_u = e_{ux} \vec{e}_x + e_{uy} \vec{e}_y + e_{uz} \vec{e}_z, $$

$$ \vec{e}_v = e_{vx} \vec{e}_x + e_{vy} \vec{e}_y + e_{vz} \vec{e}_z, $$

$$ \vec{e}_w = e_{wx} \vec{e}_x + e_{wy} \vec{e}_y + e_{wz} \vec{e}_z. $$

If $e_{wx}$, $e_{wy}$ and $e_{wz}$ are unknown you can find it by taking the cross product. Assuming that the initial basis vectors are right-handed, then $\vec{e}_w$ can be found with,

$$ \vec{e}_w = \vec{e}_u \times \vec{e}_v = (e_{uy}e_{vz} - e_{uz}e_{vy}) \vec{e}_x + (e_{uz}e_{vx} - e_{ux}e_{vz}) \vec{e}_y + (e_{ux}e_{vy} - e_{uz}e_{vx}) \vec{e}_z, $$

The rotation of the basis vectors can be calculated with the help of the rotation matrix and can be found with the help of dyadic products,

$$ R = \vec{e}_u\vec{e}_x + \vec{e}_v\vec{e}_y + \vec{e}_w\vec{e}_z, $$

this yields a second order tensor. In order to calculate the rotation you would have to take the dot product, but if you instead of dyadic products use vector direct products you can use a normal matrix-vector multiplication. In MATLAB this would can be done with,

R = eu * ex' + ev * ey' + ew * ez';

such that,

$$ \begin{bmatrix} v_u \\ v_v \\ v_w \end{bmatrix} = R \begin{bmatrix} v_x \\ v_y \\ v_z \end{bmatrix} + \begin{bmatrix} o_u \\ o_v \\ o_w \end{bmatrix}. $$

You can do it using matrix inverses. Three matrix-vector multiplications (e.g. transforming three 3D vectors by a 3x3 matrix) is equivalent to multiplying two 3x3 matrices together.

So, you can put your first set of points in one matrix, call it A:

0 0 1  < vector 1
0 1 0  < vector 2
2 0 0  < vector 3

Then put your second set of points in a second matrix, call it C. As an example, imagine a transform that scales by a factor of 2 around the origin and flips the Y and Z axes:

0 2 0  < vector 1
0 0 2  < vector 2
4 0 0  < vector 3

So, if A x B = C, we need to find the matrix B, which we can find by finding the A^-1:

Inverse of A:

0 0 0.5
0 1 0
1 0 0

The multiply A^-1 x C (in that order):

2 0 0
0 0 2
0 2 0

This is a transform matrix B that you can apply to new points. Dot-product multiply the vector by the first column to get the transformed X, second column to get the transformed Y, etc.

You should read how to change basis and think in vector, not arrays but the math ones :P

Your explanation is not clear enough for me to figure out what you've done wrong, so I'll just tell you how to do it from scratch. Follow these directions and you should be able to figure out how to solve your problem.

The key to figuring out coordinate transformations of matrices is to be very, very clear about what a linear transformation is, what a matrix is, and what the relation is. I'll start with the basics. I'll start by repeating stuff you know, so bear with that. I just need to be sure that we are on the same page with terminology.

A linear transformation is a function F that maps one vector space to another (which frequently is the same one).

Given a finite dimensional vector space and a basis, you can write down any vector in that vector space in a unique way as a set of associated coordinates. The representation you get depends very much on the basis. We usually write this as a vertical column.

Given two finite dimensional vector spaces, and a pair of associated bases a and b, and a linear transformation F between them, we can write down a matrix as follows.

1. For each a_i in a, write down F(a_i) in the basis b as a column.
2. Concatenate those columns to get a matrix.

This operation will prove critical!

Thus a matrix depends on a linear transformation and a pair of choices of basis. I will represent this as M = Fab.

Function composition is represented by matrix multiplication. That is Fab * Gbc = (F o G)ac. (The whole point of the definition of matrix multiplication is to make this true.)

Now in your case you have a linear transformation 'F' from R3 back to R3. You have it represented in a basis 'b' and you want it represented in a basis 'a'. That is you have Fbb and want to have Faa. Let I be the identity linear transformation, that just sends it to itself. Then Faa = Iab * Fbb * Iba and all that you need to do is figure out what the matrices Iab and Iba are.

Now you have your coordinate systems a (usual coordinates) and b. You know what b is written out in coordinate system a. Follow the directions above for representing a linear operator by a matrix (the one that I said would prove critical) and you can immediately write down Iba. The matrix Iab is the inverse of this one.

Note that many get confused by the fact that Iab is the inverse of what they would guess it should be. But it is something that is easy to experience. I highly encourage you to actually try this. Get up. Turn 90 degrees. Note that while you turn one way, it looks like the world turns the other. You're experiencing the fact that as your coordinate system rotates one way, the representation of things does the inverse (ie rotates the other way).

Edit: The explanation of linear operators is correct, however the stated problem involves rotations around a point that is not the origin. All linear operators send the origin to the origin. This kind of operation can send the origin elsewhere. Therefore we need to move from linear functions to affine ones. But luckily affine functions are not a big complication. They are literally a constant point plus a linear piece. That is instead of functions that can be represented by Mx with M a matrix, you wind up with functions of the form p + Mx with p a point and M a matrix.

First it is very helpful to introduce an important but seemingly arbitrary distinction. A point is a point. A vector is an offset between points, which can be represented as one point minus another. Points and vectors really are different kinds of things and we should think of them differently.

Here is the reason we make the distinction. If F is an affine function on points, then F is a linear function mapping vectors to vectors. (The constant bit moves both the start and end points of the vector, but doesn't change how much they are offset from each other.) That lets us easily separate the constant and linear parts of the affine function.

When you're dealing with affine functions, there is no particular reason to only deal with coordinate systems centered at the origin, and often there are good reasons not to. Therefore a coordinate system is determined by a point p and a basis a for the vector space. (Slightly more complicated, but not much.) In that coordinate system an arbitrary point q can be mapped to the vector q - p and then that vector represented in the basis a. Again it is customary to write those coordinates as a column.

Now if F is an affine function, then all you need to do is figure out the linear part of F, and add in where F sends the origin of your coordinate system.

If you're going to take an affine function in one coordinate system, (p, a), and transform it to another, (q, b), all that you do is transform the linear bit from the basis a to the basis b, and then figure out what F(p) works out to be.

So affine functions are basically linear functions with information about where the origin goes added in. Most of the work is spent on the matrix representation of the linear function, but you can't forget figuring out what happens to the origin.