There are many ways to solve the problem. The conceptually simplest, but most tedious, is to test one by one the possibilities $x\equiv 0\pmod{5}$, $x\equiv 1\pmod{5}$, and so on up to $x\equiv 4\pmod{5}$. Quickly we find that $x\equiv 4\pmod{5}$. (This approach would become quite unpleasant if $5$ were replaced by $97$.)

It is simpler to use some algebra. So rewrite as $2x\equiv 3\pmod{5}$. Since $3\equiv 8\pmod{5}$, it is convenient to rewrite the congruence as $2x\equiv 8\pmod{5}$. Then since $2$ and $5$ are relatively prime, we can divide by $2$, getting $x\equiv 4\pmod{5}$.

A fancier version is to start from $2x\equiv 3\pmod{5}$. Now multiply both sides by $3$ (the modular inverse of $2$). We get $6x\equiv 9\pmod{5}$. But $6\equiv 1\pmod{5}$ and $9\equiv 4\pmod{5}$, so we conclude that $x\equiv 4\pmod{5}$.

Remark: We used congruence notation throughout, since it is very important to get accustomed to it. But $2x\equiv 3\pmod{5}$ means that $5$ divides $2x-3$. So we want to solve $2x-3=5k$, that is, $2x=3+5k$. So we want to find a $k$ such that $3+5k$ is divisible by $2$. It is clear that $k=1$ works, giving $2x=8$ so $x=4$. Any number congruent to $4$ modulo $5$ will also work, giving answer $x\equiv 4\pmod{5}$.

Mathematically, congruence modulo $n$ is an equivalence relation. We define:

$$a\equiv b \pmod n \iff n\mid (a - b)$$

Equivalently: When working in $\pmod n$, any number $a$ is congruent $\mod n$ to an integer $b$ if there exists an integer $k$ for which $\;nk = (a - b)$.

Now, let's compare the "discrepancies" in the equivalences you note (which are, in fact, all true):

$$13 \equiv \color{blue}{\bf 1} \pmod 4 \iff 4\mid (13-1) \iff 4\mid 12\; \checkmark$$ $$13\equiv \color{blue}{\bf 5} \pmod 4 \iff 4\mid (13-5) \iff 4\mid 8 \;\checkmark$$

• Note, indeed, that $\color{blue}{\bf 5 \equiv 1} \pmod 4$ since $4\mid(5 - 1) = 4$ $$ $$

$$9 \equiv \color{blue}{\bf 4} \pmod 5 \iff 5 \mid (9-4) \iff 5\mid 5 \;\checkmark$$ $$9 \equiv \color{blue}{\bf -1} \pmod 5 \iff 5 \mid (9 - (-1)) \iff 5\mid 10 \;\checkmark$$

• And again, note that $\color{blue}{\bf4 \equiv -1} \pmod 5$ since $5\mid(4-(-1)) = 5$ $$ $$

It is often customary to express equivalence modulo $n$, by choosing $b$ in $\;a \equiv b \pmod n\;$ to be such that $0 \leq b \lt n$. But this choice is simply a representative of all the numbers which belong to the same equivalence class, denoted $[b]$, $\pmod n$:

E.g. If $n = 4$, then one of the following holds: $$a \equiv b \pmod 4 \iff \begin{cases} a, b \in [0] = \{4k + 0\mid k\in \mathbb Z\} = \{\cdots, -8, -4, 0, 4, 8, 12,\cdots\} \\ \\ a, b \in [1] = \{4k + 1\mid k \in \mathbb Z\} = \{\cdots, -7, -3, 1, 5, 9, 13,\cdots\} \\ \\ a ,b \in [2] = \{4k + 2\mid k \in \mathbb Z\} = \{\cdots, -6, -2, 2, 6, 10, 14,\cdots\} \\ \\ a, b \in [3] = \{4k + 3\mid k \in \mathbb Z\} = \{\cdots, -5, -1, 3, 7, 11, 15, \cdots\} \end{cases} $$

That depends on your definition of the remainder, which in turn depends on a definition of 'integer division'.

It's quite easy for positive numbers: the result of division is the largest integer not exceeding the exact result. For example 5/8 = 0. Then the remainder is 5–8*(8/5) = 5–8*0 = 5.

For negative numbers, however, a problem appears with a meaning of 'the largest'. One can assume it is the value largest with respect to its absolute value, i.e. the result is rounded towards zero (some programming languages work this way); then the integer division (–5)/8 results in –0=0, and the remainder is –5.
Or one can take literally the largest value, in which case (–5)/8=–1 and then the remainder is 3.