A norm is a function that takes a vector as an input and returns a scalar value that can be interpreted as the "size", "length" or "magnitude" of that vector. More formally, norms are defined as having the following mathematical properties:

• They scale multiplicatively, i.e. Norm(a·v) = |a|·Norm(v) for any scalar a
• They satisfy the triangle inequality, i.e. Norm(u + v) ≤ Norm(u) + Norm(v)
• The norm of a vector is zero if and only if it is the zero vector, i.e. Norm(v) = 0 ⇔ v = 0

The Euclidean norm (also known as the L² norm) is just one of many different norms - there is also the max norm, the Manhattan norm etc. The L² norm of a single vector is equivalent to the Euclidean distance from that point to the origin, and the L² norm of the difference between two vectors is equivalent to the Euclidean distance between the two points.

As @nobar's answer says, np.linalg.norm(x - y, ord=2) (or just np.linalg.norm(x - y)) will give you Euclidean distance between the vectors x and y.

Since you want to compute the Euclidean distance between a[1, :] and every other row in a, you could do this a lot faster by eliminating the for loop and broadcasting over the rows of a:

dist = np.linalg.norm(a[1:2] - a, axis=1)

It's also easy to compute the Euclidean distance yourself using broadcasting:

dist = np.sqrt(((a[1:2] - a) ** 2).sum(1))

The fastest method is probably scipy.spatial.distance.cdist:

from scipy.spatial.distance import cdist

dist = cdist(a[1:2], a)[0]

Some timings for a (1000, 1000) array:

a = np.random.randn(1000, 1000)

%timeit np.linalg.norm(a[1:2] - a, axis=1)
# 100 loops, best of 3: 5.43 ms per loop

%timeit np.sqrt(((a[1:2] - a) ** 2).sum(1))
# 100 loops, best of 3: 5.5 ms per loop

%timeit cdist(a[1:2], a)[0]
# 1000 loops, best of 3: 1.38 ms per loop

# check that all 3 methods return the same result
d1 = np.linalg.norm(a[1:2] - a, axis=1)
d2 = np.sqrt(((a[1:2] - a) ** 2).sum(1))
d3 = cdist(a[1:2], a)[0]

assert np.allclose(d1, d2) and np.allclose(d1, d3)

OK let's see if this helps you. Suppose you have two functions $f,g:[a,b]\to \mathbb{R}$. If someone asks you what is distance between $f(x)$ and $g(x)$ it is easy you would say $|f(x)-g(x)|$. But if I ask what is the distance between $f$ and $g$, this question is kind of absurd. But I can ask what is the distance between $f$ and $g$ on average? Then it is $$ \dfrac{1}{b-a}\int_a^b |f(x)-g(x)|dx=\dfrac{||f-g||_1}{b-a} $$ which gives the $L^1$-norm. But this is just one of the many different ways you can do the averaging: Another way would be related to the integral $$ \left[\int_a^b|f(x)-g(x)|^p dx\right]^{1/p}:=||f-g||_{p} $$ which is the $L^p$-norm in general.

Let us investigate the norm of $f(x)=x^n$ in $[0,1]$ for different $L_p$ norms. I suggest you draw the graphs of $x^{p}$ for a few $p$ to see how higher $p$ makes $x^{p}$ flatter near the origin and how the integral therefore favors the vicinity of $x=1$ more and more as $p$ becomes bigger. $$ ||x||_p=\left[\int_0^1 x^{p}dx\right]^{1/p}=\frac{1}{(p+1)^{1/p}} $$ The $L^p$ norm is smaller than $L^m$ norm if $m>p$ because the behavior near more points is downplayed in $m$ in comparison to $p$. So depending on what you want to capture in your averaging and how you want to define `the distance' between functions, you utilize different $L^p$ norms.

This also motivates why the $L^\infty$ norm is nothing but the essential supremum of $f$; i.e. you filter everything out other than the highest values of $f(x)$ as you let $p\to \infty$.

Well, while it is true that under some noation $\nabla_x \|x\|^2_2=2x$, I think it is a bad habit as it doesn't go natuarlly with matrix calculus. Usually, given $f:\mathbb R^n\rightarrow\mathbb R^m$, its derivative matrix $Df$ is an $m\times n$ matrix. The $ij$ entry is the derivative of $f_i$ w.r.t $x_j$. So in this case we expect $D(\|x\|^2_2)$ to be an $1\times n$ row vector. Namely, $D(\|x\|^2_2)=2x^T$.

The chain rule only works naturally in this setting, when we are using derivative matrices. It says that (under some simplification), $D(f\circ g) (x)= Df(g(x))\cdot Dg(x)$ The multiplication is matrix multiplication. Under this setting we see the following: I am using the well known formula $D(Ax)=A$ (which makes sense, a linear approximation of a linear function is itself) $$D_x(\|z-Zx\|_2^2)=2(z-Zx)^T D(z-Zx)=-2(z^T-x^TZ^T)Z$$ This is a row vector of course. I am guessing your book equate it to zero. In this case, to get a nicer notation you may transpoe both sides and get $Z^T(z-Zx)=0$. Other approaches would be using the fact that the squared norm is just $(Zx-z)^T(Zx-z)$, foiling and then there's no need to use the chain rule.