Doing everything in one heredoc in the SCL environment is the best option, IMO:

scl enable python27 - << \EOF
cd /var/www/python/scripts/
python runAllUpserts.py >/dev/null 2>&1
EOF

Another way is to run just the second command (which is the only one that uses Python) in scl environment directly:

cd /var/www/python/scripts/
scl enable python27 "python runAllUpserts.py >/dev/null 2>&1"
Answer from slavek on Stack Overflow
🌐
Linux Man Pages
linux.die.net › man › 1 › scl
scl(1) - Linux man page
scl <action> [<collection1> <collection2> ...] <command>
🌐
Unix.com
unix.com › shell programming and scripting
How to use scl enable python command call with in bash shell script? - Shell Programming and Scripting - Unix Linux Community
January 24, 2019 - I have a bash shell script, within ... use the below two lines within bash shell script. scl enable rh-python34 python 'excel2csv.py' '-P' 'Khai123' '/usr/local/bin/ltchadoop......
🌐
Gitbook
uaf-rcs.gitbook.io › uaf-rcs-hpc-docs › third-party-software › using-software-collections-scl
Using Software Collections (SCL) | UAF-RCS HPC Documentation
March 28, 2020 - For example, to start a session with python version 3.6.X: ... When in an SCL session, the exit command will end the session. A second exit command will have to be executed to end the login session. In order to use an SCL in a batch job, the job script either has to launch using scl to load the desired software, or the job script has to execute another script that does so. To launch a bash script (including a job script) that loads the python 3.6.X SCL, for example, put the following as the first line.
🌐
Red Hat
access.redhat.com › solutions › 1252913
How to use Red Hat Software Collections in scripts and cron jobs - Red Hat Customer Portal
Setup cron calling the bash wrapper script 30 * * * * scl enable python27 <path to script>/bash_script.sh
🌐
SysTutorials
systutorials.com › docs › linux › man › 1-scl
scl: Setup and run software from Software Collection environment - Linux Manuals (1)
<path> directory needs to contain enable scriptlet and root directory, to be considered valid SCL. <path> needs to be an absolute path to the collection location. ... <collection> will no longer be considered SCL. If the <collection> was installed locally, then the use of --force is needed.i ... run set of commands listed in my_command file in the environment with baz Software Collection enabled ... list all packages within example collection scl register /foo/bar registers new collection with a name bar
Find elsewhere
🌐
Red Hat
access.redhat.com › solutions › 527703
How can I make a Red Hat Software Collection persist after a reboot/logout? - Red Hat Customer Portal
With that package, you can enable userspace environment automatically by creating a new file under /etc/profile.d with the following content (example for python33 collection): $ cat /etc/profile.d/enablepython33.sh #!/bin/bash source scl_source enable python33 · With such file all users will have python 3.3 collection enabled by default in shell. NOTE: However, there is a side effect with this approach in that you cannot disable the collections thereafter. Also, shebang in the script affects which interpreter will be used.
🌐
Unix.com
unix.com › man_page › centos › 1 › scl
scl(1) centos man page | unix.com
EXAMPLES scl enable example 'less --version' runs command 'less --version' in the environment with collection 'example' enabled scl enable foo bar bash runs bash instance with foo and bar Software Collections enabled cat my_command | scl enable ...
Top answer
1 of 1
3

I'm not familiar with scl at all, but I can point out some things that're wrong with the shell part of what you're doing:

  • Use $ with variable to get their values, not when setting them. Therefore, $ARG=argument should be ARG=argument. BTW, the error message you gave doesn't actually match this problem; it looks more like what you'd get with spaces around the = (e.g. ARG = argument), which isn't allowed because it'll treat ARG as a command, and "=" and "argument" as arguments to pass to it. In general, spaces are important delimiters in the shell, and adding or removing them -- even in unimportant-looking places -- can completely change the meaning of a command.

  • By default, variables are private to the shell itself. To make the variable available to scl (and any other commands you run in the shell), you need to export it. Thus, you actually want:

    Copyexport ARG=argument
    
  • I recommend against using all-caps variable names, since there are a number of all-caps names that have special meanings; using one of those for something else can have weird effects. Lower- and mixed-case variable names are safer.

  • Inside a regular here-document (the part that << SS opens), the shell will expand $variable expressions before sending the document to the program. If you want that $ expression to be sent to the program for it to interpret, you need to either quote the here-doc delimiter (e.g. << "SS"), or backslash-escape all dollar-signs, backticks, and backslash characters in the document.

Top answer
1 of 3
3

You should try using -- instead of surrounding your command with quotes.

scl enable python27 -- ls /tmp

I was able to make a python script that uses the rh-python35 collection with this shebang:

#!/usr/bin/scl enable rh-python35 -- python

import sys
print(sys.version)
2 of 3
1

The parsing of arguments in the she-bang command is not really defined. From man execve:

The semantics of the optional-arg argument of an interpreter script vary across implementations. On Linux, the entire string following the interpreter name is passed as a single argument to the interpreter, and this string can include white space. However, behavior differs on some other systems. Some systems use the first white space to terminate optional-arg. On some systems, an interpreter script can have multiple arguments, and white spaces in optional-arg are used to delimit the arguments.

No matter what, argument splitting based on quote sis not supported. So when you write:

#!/usr/bin/scl enable python27 "ls /tmp"

It's very possible that what gets invoked is (using bash notation):

'/usr/bin/scl' 'enable' 'python27' '"ls' '/tmp"'

This is probably why it tries to open the "ls file at /etc/scl/prefixes/"ls

But it is just as likely that the shebang evaluates to:

'/usr/bin/scl' 'enable python27 "ls /tmp"'

And that would fail since it wont be able to find a command named enable python27 "ls /tmp" for scl to execute.

There's a few workarounds you can use.

You can call your script via scl:

$ cat myscript
#!/bin/bash
echo hello

$ scl enable python27 ./myscript
hello

You can also use the heredoc notation, but it might lead to subtle issues. I personally avoid this:

$ cat ./myscript
#!/bin/bash
scl enable python27 -- <<EOF
echo hi
echo \$X_SCLS
EOF

$ bash -x myscript 
+ scl enable python27 --
hi
python27

You can see one of the gotcha's already: I had to write \$X_SCLS to access the environment variable instead of just $X_SCL.

Edit: Another option is two have two scripts. One that has the actual code, and the second that simply does scl enable python27 $FIRST_SCRIPT. Then you wont have to remember to enter scl ... manually.

🌐
ManKier
mankier.com › home › scl-utils
scl: Setup and run software from Software Collection environment | Man Page | Commands | scl-utils | ManKier
If the <collection> was installed ... environment with baz Software Collection enabled ... #!/bin/bash COMMAND="ruby $(printf "%q " "$@")" scl enable ruby193 "$COMMAND" # or scl enable ruby193 -- $COMMAND...
🌐
GitHub
github.com › sclorg › scl-utils › blob › master › doc › scl.1
scl-utils/doc/scl.1 at master · sclorg/scl-utils
A wrapper script must use proper quoting or command separator like in the · following examples · .PP · .nf · .RS · #!/bin/bash · COMMAND="ruby $(printf "%q " "$@")" scl enable ruby193 "$COMMAND" # or · scl enable ruby193 -- $COMMAND · ...
Author   sclorg
🌐
Georgia Tech
support.cc.gatech.edu › support-tools › howto › how-use-software-collections
How to Use Software Collections | Technology Services Organization
To run the Bash shell in the environment with multiple Software Collections enabled, execute the following command: scl enable software_collection_1 software_collection_2 bash
Top answer
1 of 5
93

In your ~/.bashrc or ~/.bash_profile Simply source the "enable" script provided with the devtoolset. For example, with the Devtoolset 2, the command is:

source /opt/rh/devtoolset-2/enable

or

source scl_source enable devtoolset-2

Lot more efficient: no forkbomb, no tricky shell

2 of 5
17

An alternative of source /opt/rh/devtoolset-4/enable is

source scl_source enable devtoolset-4

The above shell script scl_source is more elegant than using a hard coded path (may be different on another machine). However scl_source does less because /opt/rh/devtoolset-4/enable uses scl_source and other stuff.

To use scl_source you may have to upgrade package scl-utils

yum update scl-utils  # old scl-utils versions miss scl_source

Quick copy-paste

echo 'source scl_source enable devtoolset-4' >> ~/.bashrc
    # Do not forget to change the version ↑

Source code for curious people

An example of scl_source source code:
https://gist.github.com/bkabrda/6435016

The scl_source installed on my Red Hat 7.1

#!/bin/bash

_scl_source_help="Usage: source scl_source <action> [<collection> ...]

Don't use this script outside of SCL scriptlets!

Options:
    -h, --help    display this help and exit"

if [ $# -eq 0 -o $1 = "-h" -o $1 = "--help" ]; then
    echo "$_scl_source_help"
    return 0
fi


if [ -z "$_recursion" ]; then
    _recursion="false"
fi
if [ -z "$_scl_scriptlet_name" ]; then
    # The only allowed action in the case of recursion is the same
    # as was the original
    _scl_scriptlet_name=$1
fi
shift 1

if [ -z "$_scl_dir" ]; then
    # No need to re-define the directory twice
    _scl_dir=/etc/scl/conf
    if [ ! -e $_scl_dir ]; then
        _scl_dir=/etc/scl/prefixes
    fi
fi

for arg in "$@"; do
    _scl_prefix_file=$_scl_dir/$arg
    _scl_prefix=`cat $_scl_prefix_file 2> /dev/null`
    if [ $? -ne 0 ]; then
        echo "Can't read $_scl_prefix_file, $arg is probably not installed."
        return 1
    fi

    # First check if the collection is already in the list
    # of collections to be enabled
    for scl in ${_scls[@]}; do
        if [ $arg == $scl ]; then
            continue 2
        fi
    done

    # Now check if the collection isn't already enabled
    /usr/bin/scl_enabled $arg > /dev/null 2> /dev/null
    if [ $? -ne 0 ]; then
        _scls+=($arg)
        _scl_prefixes+=($_scl_prefix)
    fi;
done

if [ $_recursion == "false" ]; then
    _i=0
    _recursion="true"
    while [ $_i -lt ${#_scls[@]} ]; do
        _scl_scriptlet_path="${_scl_prefixes[$_i]}/${_scls[$_i]}/${_scl_scriptlet_name}"
        source "$_scl_scriptlet_path"
        if [ $? -ne 0 ]; then
            echo "Can't source $_scl_scriptlet_name, skipping."
        else
            export X_SCLS="${_scls[$_i]} $X_SCLS"
        fi;
        _i=$(($_i+1))
    done
    _scls=()
    _scl_prefixes=()
    _scl_scriptlet_name=""
    _recursion="false"
fi
🌐
Soton
linuxdesktops.soton.ac.uk › softwarecollections.html
Software collections (SCL) — Linux Desktops RHEL7 documentation
Once you have installed an SCL package you must first ‘enable’ its use and run a program with the package enabled. For example: ... This ‘enables’ the rh-python35 package (Python 3.5) and then runs the command bash.