Error: 'rootDir' is expected to contain all source files.

rootDir is expected to be the root of all source/input files. TS compiler proceeds like this:

collect all input files ("files" / "include" / "exclude" / imports) 
  --> for each included input file
    --> chop off the "rootDir" from the input 
    --> prepend the "outDir"

The src/test files import from src/main, so currently rootDir can only be set to src or above.

Solution: Project References (TS 3.0+)

Project references can solve all of the issues you described:

• rootDir will be set to src/main or src/test respectively
• src folder does not appear as part of the lib output
• src/main cannot import src/test (only vice versa should be possible)

|   tsconfig-base.json            // other config options go here
|   tsconfig.json                 // solution config containing references
|   
+---lib                           // output folder
|   +---main
|   |       mod.d.ts
|   |       mod.js
|   |       tsconfig.tsbuildinfo
|   |       
|   \---test
|           mod-test.js     
\---src
    +---main                      // referenced sub-project main 
    |       mod.ts
    |       tsconfig.json
    |       
    \---test                      // referenced sub-project test 
            mod-test.ts
            tsconfig.json

{
  "compilerOptions": {
    // just some sample values, set your own as needed
    "target": "ESNext",
    "module": "ESNext",
    "moduleResolution": "node",
    "strict": true,
    "esModuleInterop": true,
    // other compiler options
  }
}

{
  "files": [],
  "references": [
    {
      "path": "./src/main"
    },
    {
      "path": "./src/test"
    }
  ]
}

export const foo = "foo";

{
  "extends": "../../tsconfig-base.json",
  "compilerOptions": {
    "outDir": "../../lib/main",
    "composite": true // flag to signal referenced project      
  }
}

import { foo } from "../main/mod";

// use your test framework here
export function testMod() {
  if (foo !== "foo") throw new Error("foo expected.");
}

testMod()

{
  "extends": "../../tsconfig-base.json",
  "compilerOptions": {
    "outDir": "../../lib/test"
  },
  "references": [
    {
      "path": "../main" // test project depends on main
    }
  ]
}

"scripts": {
    "build": "tsc -b -v",
    "clean": "tsc -b --clean"
},

Further links

• TypeScript Project References Demo

According to tsc --help:

  --project, -p  Compile the project given the path to its configuration file, or to a folder with a 'tsconfig.json'.

  --build, -b  Build one or more projects and their dependencies, if out of date

The --project option compile a single project.

The --build option can be seen as a build orchestrator that find referenced projects, check if they are up-to-date, and build out-of-date projects in the correct order. See the documentation for details.

To answer your second question, the --build option is slower because it also compiles dependencies, but it should be faster at a second run, because it compiles only out-of-date projects.