R has lexical scope; a variable that is not assigned in a function is resolved against its static environment. In your case, f1 sees the value of c in your global environment, which is 2. Within f2, c == 3 because it is passed as an argument which masks the global c, but when f1 is called, it still sees c==2 from its enclosing environment. You expected dynamic scoping, where c would take the value of its runtime environment (i.e., within f2). You can set c <- 3 outside the functions, before f2, or you can do this to copy the local value passed as 'c' in f2 to the global environment:

 f2 = function(a,b,c){
    c <<- c     # local argument c copied to global c
    f1(a,b)
 }

more discussion of lexical scope in R: https://darrenjw.wordpress.com/2011/11/23/lexical-scope-and-function-closures-in-r/

R uses lexical scoping which means that if a function needs to reference an object not defined in that function it looks at the environment in which the function was defined, not the caller. In the question g is defined in the global environment so that is where g looks for c.

Also note that in R we would not call the functions in the question nested functions. Rather what is nested is the calls, not the functions. In (3) below we show nested functions.

1) We can reset a function's environment in which case it will think it was defined in that environment.

g <- function(a,b){       # help function
    a^2 + b^2 - c
}

f <- function(a,b,c,d){   # main function
    environment(g) <- environment()
    g(a,b)
}

f(1, 2, 3, 4)
## [1] 2

2) Another possibility is to explicitly tell it which environment to search using envir$c (where envir is the desired environment) or get("c", envir) or with(envir, c) . envir$c will look into envir. The other two will look there and if not found will look into ancestor environments. (Each environment has a parent or the emptyenv(). This is distinct from the call stack.)

g <- function(a, b, envir = parent.frame()){       # help function
    a^2 + b^2 - envir$c
}

f <- function(a,b,c,d){   # main function
    g(a,b)
}

f(1, 2, 3, 4)
## [1] 2

3) We can nest the functions so that g is defined in f.

f <- function(a,b,c,d){   # main function
    g <- function(a,b){       # help function
        a^2 + b^2 - c
    }
    g(a,b)
}

f(1, 2, 3, 4)
## [1] 2

4) Of course you could just pass c and avoid all these problems.

g <- function(a, b, c) { 
    a^2 + b^2 - c
}

f <- function(a, b, c, d) {   # main function
    g(a, b, c)
}

f(1, 2, 3, 4)
## [1] 2

When an expression is on a line by itself it will automatically print if you do it at the R console but that does not happen if it is within a function or within an expression. Use cat or print for displaying.

To return two objects return a list containing both of them as shown at below.

The value of the last line that is run in a function is returned so you rarely need return.

f1a <- function(x, y) {
    f2 <- function(a, b) {
        print("f2")
        a + b
    }
    print("f1")
    list(x - y, f2(x, y))
}

result <- f1a(8, 5)
## [1] "f1"
## [1] "f2"

result[[1]]
## [1] 3
result[[2]]
## [1] 13

result
## [[1]]
## [1] 3
##
## [[2]]
## [1] 13

Other things we could do would be to replace the list(...) line in the code above with one of the following. (The c versions would only be used if we knew that the arguments were always scalars.)

list(f1 = x - y, f2 = f2(x, y))       # named list
c(x - y, f2(x, y))                    # 2 element numeric vector
c(f1 = x - y, f2 = f2(x, y))          # 2 element named numeric vector
cbind(f1 = x - y, f2 = f2(x, y))      # matrix w column names
data.frame(f1 = x - y, f2 = f2(x, y)) # data.frame