If you don't insist on passing something that looks like a list, but simply want to pass multiple variadic arguments, you can use the multiple option.

From the click documentation

@click.command()
@click.option('--message', '-m', multiple=True)
def commit(message):
    click.echo('\n'.join(message))

$ commit -m foo -m bar
foo
bar

The simplest solution for single arguments or options seems to be to just create a variable ARG = click.option(...) and then use @ARG as a decorator of the command function.

Applying multiple opts/args at once involves creating a decorator factory or a callable class. This is the cleanest solution I found, credits go to @altendky and @SnoopJ from #python channel on Libera IRC:

from typing import List, Callable

import attr
import click

from click.decorators import FC

options = [
    click.option("-a"),
    click.option("-b"),
]


# Solution 1: decorator factory
def multi_decorator(options: List[Callable[[FC], FC]]) -> Callable[[FC], FC]:
    def decorator(f: FC) -> FC:
        for option in reversed(options):
            f = option(f)

        return f

    return decorator


@click.command()
@multi_decorator(options)
def cmd1(a, b):
    pass


# Solution 2: callable class
@attr.s
class MultiDecorator:
    options: List[Callable[[FC], FC]] = attr.ib(factory=list)

    def __call__(self, f: FC):
        for option in reversed(self.options):
            f = option(f)

        return f


@click.command()
@MultiDecorator(options)
def cmd2(a, b):
    pass


if __name__ == "__main__":
    from click.testing import CliRunner
    runner = CliRunner()

    print(runner.invoke(cmd1, ["--help"]).output)
    print(runner.invoke(cmd2, ["--help"]).output)

If you want to test or improve this, head over to https://replit.com/@ldericher/multidecorator.

import sys
import click
import pandas as pd
from sklearn import metrics
@click.command()

# @click.option('--filename')
## @click.argument('result.csv', type=click.Path(exists=True))
@click.argument('filename', type=click.Path(exists=True))
def main(filename):

    df = pd.read_csv(filename)
    y_test = df["actual"].values
    y_pred = df["predicted"].values

    print('R Squared: ' +  str(metrics.r2_score(y_test, y_pred)))
    
if __name__ == '__main__':
    # sys.argv = ['',  '--filename', 'result.csv']
    sys.argv = ['',  'result.csv']
    main()

The key to making this work is to save the removedigits value into the context with something like:

Code:

@click.pass_context
def cli(ctx, removedigits):
    ctx.obj = dict(removedigits=removedigits)

Then you can retreive it like:

@click.pass_context
def upper(ctx, arguments):
    click.echo(ctx.obj['removedigits'])

Test Code:

import click


@click.group()
@click.option('--removedigits/--removenodigits', default=False,
              help="To remove digits or not")
@click.pass_context
def cli(ctx, removedigits):
    ctx.obj = dict(removedigits=removedigits)


def removedigits(ctx, a_string):
    if ctx.obj['removedigits']:
        a_string = ''.join(c for c in a_string if not c.isdigit())
    return a_string


@cli.command()
@click.argument('arguments', nargs=-1)
@click.pass_context
def upper(ctx, arguments):
    click.echo(ctx.obj['removedigits'])
    click.echo(removedigits(ctx, ' '.join(arguments).upper()))


@cli.command()
@click.argument('arguments', nargs=-1)
@click.pass_context
def lower(ctx, arguments):
    click.echo(ctx.obj['removedigits'])
    click.echo(removedigits(ctx, ' '.join(arguments).lower()))


if __name__ == '__main__':
    commands = (
        '--removedigits lower Hi Mom',
        '--removedigits upper Hi Mom',
        '--removedigits upper Hi1 Mom2',
        '--removenodigits upper Hi Mom',
        '--removenodigits upper Hi1 Mom2',
        'upper Hi Mom',
        'upper Hi1 Mom2',
        '--help',
    )

    import sys, time
    time.sleep(1)
    print('Click Version: {}'.format(click.__version__))
    print('Python Version: {}'.format(sys.version))
    for cmd in commands:
        try:
            time.sleep(0.1)
            print('-----------')
            print('> ' + cmd)
            time.sleep(0.1)
            cli(cmd.split())

        except BaseException as exc:
            if str(exc) != '0' and \
                    not isinstance(exc, (click.ClickException, SystemExit)):
                raise

Results:

Click Version: 6.7
Python Version: 3.6.3 (v3.6.3:2c5fed8, Oct  3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)]
-----------
> --removedigits lower Hi Mom
True
hi mom
-----------
> --removedigits upper Hi Mom
True
HI MOM
-----------
> --removedigits upper Hi1 Mom2
True
HI MOM
-----------
> --removenodigits upper Hi Mom
False
HI MOM
-----------
> --removenodigits upper Hi1 Mom2
False
HI1 MOM2
-----------
> upper Hi Mom
False
HI MOM
-----------
> upper Hi1 Mom2
False
HI1 MOM2
-----------
> --help
Usage: test.py [OPTIONS] COMMAND [ARGS]...

Options:
  --removedigits / --removenodigits
                                  To remove digits or not
  --help                          Show this message and exit.

Commands:
  lower
  upper

This is possible since Click 4.0. You need to tell click to ignore the unknown option, and that you want the options gathered up and passed in. Here is your example with the relevant changes made.

Code:

@click.group()
@click.option('--arg')
def cli(arg):
    click.echo('cli arg: %s' % arg)
    pass

@cli.command(context_settings=dict(
    ignore_unknown_options=True,
    help_option_names=[],
))
@click.argument('script')
@click.argument('args', nargs=-1, type=click.UNPROCESSED)
def run(script, args):
    click.echo('script: %s' % script)
    click.echo(args)

Test:

cli('--arg 3 run foo.py --arg 5 arg1 --arg2 --arg 4 --help'.split())

Produces:

cli arg: 3
script: foo.py
('--arg', '5', 'arg1', '--arg2', '"--arg', '4"', '--help')

(Source)

One way that I have managed to achieve this behaviour was by actually using arguments as below. I'll post this as a workaround, while I try to see if it could be done with an option, and I'll update my post accordingly

@click.command(context_settings={"ignore_unknown_options": True})
@click.argument("options", nargs=-1)
def myscript(options):
    option = False
    if options is ():
        option = False
    if '-o' in options or '--option' in options:
        option = True
    if len(options) > 1:
        option = options[1]
    print(option)

Later Edit Using an option, I have managed to achieve this by adding an argument to the command definition.

@click.command()
@click.option('-o', '--option', is_flag=True, default=False, is_eager=True)
@click.argument('value', nargs=-1)
def myscript(option, value):
    if option and value != ():
        option = value[0]
    print(option)

The nargs can be removed if you only expect at most one argument to follow, and can be treated as not required.

@click.command()
@click.option('-o', '--option', is_flag=True, default=False, is_eager=True)
@click.argument('value', required=False)
def myscript(option, value=None):
    if option and value is not None:
        option = value
    print(option)

It might also be possible by putting together a context generator and storing some state, but that seems the least desirable solution, since you would be relying on the context storing your state.