My understanding is that this is actually very simple:

• Javascript is always pass by value, but when a variable refers to an object (including arrays), the "value" is a reference to the object.
• Changing the value of a variable never changes the underlying primitive or object, it just points the variable to a new primitive or object.
• However, changing a property of an object referenced by a variable does change the underlying object.

So, to work through some of your examples:

function f(a,b,c) {
    // Argument a is re-assigned to a new value.
    // The object or primitive referenced by the original a is unchanged.
    a = 3;
    // Calling b.push changes its properties - it adds
    // a new property b[b.length] with the value "foo".
    // So the object referenced by b has been changed.
    b.push("foo");
    // The "first" property of argument c has been changed.
    // So the object referenced by c has been changed (unless c is a primitive)
    c.first = false;
}

var x = 4;
var y = ["eeny", "miny", "mo"];
var z = {first: true};
f(x,y,z);
console.log(x, y, z.first); // 4, ["eeny", "miny", "mo", "foo"], false

Example 2:

var a = ["1", "2", {foo:"bar"}];
var b = a[1]; // b is now "2";
var c = a[2]; // c now references {foo:"bar"}
a[1] = "4";   // a is now ["1", "4", {foo:"bar"}]; b still has the value
              // it had at the time of assignment
a[2] = "5";   // a is now ["1", "4", "5"]; c still has the value
              // it had at the time of assignment, i.e. a reference to
              // the object {foo:"bar"}
console.log(b, c.foo); // "2" "bar"

JavaScript doesn't have "reference variables" or "alias variables" "variable aliases"¹. All JS variables are independent, and they can hold arbitrary values of any type.

When we talk about references in JS, we mean reference values. Consider them to work like C++ shared_ptrs (except that garbage collection does not work by refcounting).

It does not point to another variable, but to its value

Kinda, yes. Both variables hold a reference value that references the same object. The assignment b = a created a copy of the reference value. Reassigning a to a different value (whether it is a different reference, or a primitive value like a number or null) will not affect b.

So 'a' and 'b' are just Aliases for the same variable.

No. At best, they are aliases for the same object, since they both reference the same memory location where the { a: 4 } is stored. That memory location is not a variable though, it's allocated in managed memory.

1: Module imports are real aliases though. When you import { example as local } from …, then local identifier resolves to the same variable (memory location) as example in the module that exports it. Changing the value of the example variable is reflected in local. The local import cannot be assigned to, though.