R uses lexical scoping which means that if a function needs to reference an object not defined in that function it looks at the environment in which the function was defined, not the caller. In the question g is defined in the global environment so that is where g looks for c.

Also note that in R we would not call the functions in the question nested functions. Rather what is nested is the calls, not the functions. In (3) below we show nested functions.

1) We can reset a function's environment in which case it will think it was defined in that environment.

g <- function(a,b){       # help function
    a^2 + b^2 - c
}

f <- function(a,b,c,d){   # main function
    environment(g) <- environment()
    g(a,b)
}

f(1, 2, 3, 4)
## [1] 2

2) Another possibility is to explicitly tell it which environment to search using envir$c (where envir is the desired environment) or get("c", envir) or with(envir, c) . envir$c will look into envir. The other two will look there and if not found will look into ancestor environments. (Each environment has a parent or the emptyenv(). This is distinct from the call stack.)

g <- function(a, b, envir = parent.frame()){       # help function
    a^2 + b^2 - envir$c
}

f <- function(a,b,c,d){   # main function
    g(a,b)
}

f(1, 2, 3, 4)
## [1] 2

3) We can nest the functions so that g is defined in f.

f <- function(a,b,c,d){   # main function
    g <- function(a,b){       # help function
        a^2 + b^2 - c
    }
    g(a,b)
}

f(1, 2, 3, 4)
## [1] 2

4) Of course you could just pass c and avoid all these problems.

g <- function(a, b, c) { 
    a^2 + b^2 - c
}

f <- function(a, b, c, d) {   # main function
    g(a, b, c)
}

f(1, 2, 3, 4)
## [1] 2

>>> def sum(x, y):
...     def do_it():
...             return x + y
...     return do_it
... 
>>> a = sum(1, 3)
>>> a
<function do_it at 0xb772b304>
>>> a()
4

Is this what you were looking for? It's called a closure.

One popular use of nested functions is closures. In a lexically scoped language with first-class functions it's possible to use functions to store data. A simple example in Scheme is a counter:

(define (make-counter)
  (let ((count 0))                ; used to store the count
    (define (counter)             ; this is the counter we're creating
      (set! count (+ count 1))    ; increment the count
      count)                      ; return the new count
    counter))                     ; return the new counter function

(define mycounter (make-counter)) ; create a counter called mycounter

(mycounter)                       ; returns 1

(mycounter)                       ; returns 2

In this example, we nest the function counter inside the function make-counter, and by returning this internal function we are able to access the data available to counter when it was defined. This information is private to this instance of mycounter - if we were to create another counter, it would use a different spot to store the internal count. Continuing from the previous example:

(define mycounter2 (make-counter))

(mycounter2)                      ; returns 1

(mycounter)                       ; returns 3

You cannot define a function within another function in standard C.

You can declare a function inside of a function, but it's not a nested function.

gcc has a language extension that allows nested functions. They are nonstandard, and as such are entirely compiler-dependent.