Programming
python argparse
Updated Fri, 30 Sep 2022 20:18:07 GMT

Why does argparse not accept "--" as argument?


My script takes -d, --delimiter as argument:

parser.add_argument('-d', '--delimiter')

but when I pass it -- as delimiter, it is empty

script.py --delimiter='--' 

I know -- is special in argument/parameter parsing, but I am using it in the form --option='--' and quoted.

Why does it not work? I am using Python 3.7.3

Here is test code:

#!/bin/python3
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('--delimiter')
parser.add_argument('pattern')
args = parser.parse_args()
print(args.delimiter)

When I run it as script --delimiter=-- AAA it prints empty args.delimiter.




Solution

This looks like a bug. You should report it.

This code in argparse.py is the start of _get_values, one of the primary helper functions for parsing values:

if action.nargs not in [PARSER, REMAINDER]:
    try:
        arg_strings.remove('--')
    except ValueError:
        pass

The code receives the -- argument as the single element of a list ['--']. It tries to remove '--' from the list, because when using -- as an end-of-options marker, the '--' string will end up in arg_strings for one of the _get_values calls. However, when '--' is the actual argument value, the code still removes it anyway, so arg_strings ends up being an empty list instead of a single-element list.

The code then goes through an else-if chain for handling different kinds of argument (branch bodies omitted to save space here):

# optional argument produces a default when not present
if not arg_strings and action.nargs == OPTIONAL:
    ...
# when nargs='*' on a positional, if there were no command-line
# args, use the default if it is anything other than None
elif (not arg_strings and action.nargs == ZERO_OR_MORE and
      not action.option_strings):
    ...
# single argument or optional argument produces a single value
elif len(arg_strings) == 1 and action.nargs in [None, OPTIONAL]:
    ...
# REMAINDER arguments convert all values, checking none
elif action.nargs == REMAINDER:
    ...
# PARSER arguments convert all values, but check only the first
elif action.nargs == PARSER:
    ...
# SUPPRESS argument does not put anything in the namespace
elif action.nargs == SUPPRESS:
    ...
# all other types of nargs produce a list
else:
    ...

This code should go through the 3rd branch,

# single argument or optional argument produces a single value
elif len(arg_strings) == 1 and action.nargs in [None, OPTIONAL]:

but because the argument is missing from arg_strings, len(arg_strings) is 0. It instead hits the final case, which is supposed to handle a completely different kind of argument. That branch ends up returning an empty list instead of the '--' string that should have been returned, which is why args.delimiter ends up being an empty list instead of a '--' string.


This bug manifests with positional arguments too. For example,

import argparse
parser = argparse.ArgumentParser()
parser.add_argument('a')
parser.add_argument('b')
args = parser.parse_args(["--", "--", "--"])
print(args)

prints

Namespace(a='--', b=[])

because when _get_values handles the b argument, it receives ['--'] as arg_strings and removes the '--'. When handling the a argument, it receives ['--', '--'], representing one end-of-options marker and one actual -- argument value, and it successfully removes the end-of-options marker, but when handling b, it removes the actual argument value.





Comments (4)

  • +1 – That begs the question, why is '--' in arg_strings in the first place? If it's acting as the optional argument terminator, shouldn't it be stripped out before that? — Aug 14, 2022 at 19:33  
  • +0 – This was reported years ago, but not fixed, bugs.python.org/issue14364 — Aug 14, 2022 at 20:18  
  • +0 – @hpaulj nice find. From that link, it seems that the PR is ready, but it's just not merged yet. — Aug 15, 2022 at 12:12  
  • +0 – The suggested patch hasn't been migrated to the github push format. I don't recall if there were outstanding issues. argparse developers try to be cautious about anything that could have backward compatibliy issues. I posted as paul.j3, but haven't done much after the github change. — Aug 15, 2022 at 15:30  


External Links

External links referenced by this document: