Hi Greg,

dict[str, str] is not a subtype of dict[str, str | something_else]
because you can assign a value of type something_else to the latter
but not the former.

I understand what you are saying but I do not yet understand why this
applies to my situation.

I don't have Python at hand currently, so I'll write untested pseudocode:

def print_greeting(greeting:int|str):
print(greeting)

print_greeting('hello')

The above snippet should be equivalent to my more complicated code over
which mypy complains to the equivalent of

"input" is of type "str"
but expected type "Union[str,int]

I do understand that "str" is formally more narrow than "Union [str,int]" and the type system has every right to not consider them equivalent.

However, this seems like a very common use case: "allow passing in either str or int
and have type checking pass either input as valid" -- yet mypy doesn't seem
to share that idea.

Or else there's something I haven't wrapped my head around yet. But what ?

Karsten


--- MBSE BBS v1.0.8.4 (Linux-x86_64)
* Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)

Dear Thomas,

thanks for taking the time to look into my issue.

Maybe it helps if I explain what I want (sorry that my web mailer does not respect
indentation, I will insert dots).

I want a function to run SQL queries:

run_queries(conn, queries):
....for q in queries:
.......conn.execute(q)

I now want to add type hints such that my large codebase can
be checked for silly doings. First, queries is to be a list
of the SQL to run:

run_queries(conn, queries:list):

Then, each list entry can be

....a string holding simple, complete SQL (say "SELECT 1")

run_queries(conn, queries:list[str]):

or

....a dict holding the SQL and arguments for parameters

run_queries(conn, queries:list[dict]):

So, taken together:

run_queries(conn, queries:list[str|dict]):

(yes, this is in Python 3.11/3.12)

Now, when it is a list of dicts I want to further constrain the
dicts. Each is to contain the keys "SQL" and "args". So the keys
are of type str. The values for the keys will be of various types,
such that I chose Any as pseudo-type, so that each list entry that
is of type dict should be dict[str, Any], hence:

queries = [{'SQL': 'SELECT %(value)s', 'args': {'value': 1}}]

and

run_queries(conn, queries:list[str|dict[str, Any]]):

If I now call this function with a simple SQL query:

SQL_query = 'SELECT 1' # should be type str ?
queries = [SQL_query] # should be type list[str] ?
run_queries(conn, queries = queries)

and run mypy over that (at least inside my complex codebase) I will
get a type mismatch being hinted at.

So far I don't grasp at which point my reasoning above is faulty.

Karsten

--- MBSE BBS v1.0.8.4 (Linux-x86_64)
* Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)

On 12/30/2023 10:08 AM, Karsten Hilbert via Python-list wrote:

Dear Thomas,

thanks for taking the time to look into my issue.

Maybe it helps if I explain what I want (sorry that my web mailer does not respect
indentation, I will insert dots).

I want a function to run SQL queries:

run_queries(conn, queries):
...for q in queries:
......conn.execute(q)

I now want to add type hints such that my large codebase can
be checked for silly doings. First, queries is to be a list
of the SQL to run:

run_queries(conn, queries:list):

Then, each list entry can be

...a string holding simple, complete SQL (say "SELECT 1")

run_queries(conn, queries:list[str]):

or

...a dict holding the SQL and arguments for parameters

run_queries(conn, queries:list[dict]):

So, taken together:

run_queries(conn, queries:list[str|dict]):

(yes, this is in Python 3.11/3.12)

Now, when it is a list of dicts I want to further constrain the
dicts. Each is to contain the keys "SQL" and "args". So the keys
are of type str. The values for the keys will be of various types,
such that I chose Any as pseudo-type, so that each list entry that
is of type dict should be dict[str, Any], hence:

queries = [{'SQL': 'SELECT %(value)s', 'args': {'value': 1}}]

and

run_queries(conn, queries:list[str|dict[str, Any]]):

If I now call this function with a simple SQL query:

SQL_query = 'SELECT 1' # should be type str ?
queries = [SQL_query] # should be type list[str] ?
run_queries(conn, queries = queries)

and run mypy over that (at least inside my complex codebase) I will
get a type mismatch being hinted at.

So far I don't grasp at which point my reasoning above is faulty.

Karsten

I am not very expert in Python type hints. In working up the example
program I just posted, I got an error message from mypy that remarked
that "list" is invariant, and to try Sequence which is "covariant". I
don't know what that means (and I haven't looked into it yet), but when
I changed from list to Sequence as suggested, mypy stopped complaining.

Here is the exact error message, and it has a reference you might want
to follow up with:

c:\temp\python\typing_test.py:16: note: "List" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance c:\temp\python\typing_test.py:16: note: Consider using "Sequence"
instead, which is covariant

Before that, mypy insisted that str and Union[str, list] were
incompatible argument types, which is something you are seeing, too.

I suggest that you build up your types as in my example, so that it's
very clear what they are and very easy to change them, and use Sequence instead of List (or list). See if that will do the job.


--- MBSE BBS v1.0.8.4 (Linux-x86_64)
* Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)

On 12/30/2023 10:08 AM, Karsten Hilbert via Python-list wrote:

Dear Thomas,

thanks for taking the time to look into my issue.

Maybe it helps if I explain what I want (sorry that my web mailer does not respect
indentation, I will insert dots).

I want a function to run SQL queries:

run_queries(conn, queries):
...for q in queries:
......conn.execute(q)

I now want to add type hints such that my large codebase can
be checked for silly doings. First, queries is to be a list
of the SQL to run:

run_queries(conn, queries:list):

Then, each list entry can be

...a string holding simple, complete SQL (say "SELECT 1")

run_queries(conn, queries:list[str]):

It occurs to me that you could simplify things if you converted those
plain query strings to dicts:

'SELECT 1' --> {'SQL': 'SELECT 1'}

I'm fairly sure your database queries don't actually give you strings or dicts, right? You probably get lists (or iterators) of tuples and
somewhere you convert them to the arguments you are feeding to
run_queries(). At least, that is how the standard Python db adapters
work. If you change that conversion step, your arguments to
run_queries() will all be lists of dicts, making your code simpler and reducing the complexity of the type hints.

or

...a dict holding the SQL and arguments for parameters

run_queries(conn, queries:list[dict]):

So, taken together:

run_queries(conn, queries:list[str|dict]):

(yes, this is in Python 3.11/3.12)

Now, when it is a list of dicts I want to further constrain the
dicts. Each is to contain the keys "SQL" and "args". So the keys
are of type str. The values for the keys will be of various types,
such that I chose Any as pseudo-type, so that each list entry that
is of type dict should be dict[str, Any], hence:

queries = [{'SQL': 'SELECT %(value)s', 'args': {'value': 1}}]

and

run_queries(conn, queries:list[str|dict[str, Any]]):

If I now call this function with a simple SQL query:

SQL_query = 'SELECT 1' # should be type str ?
queries = [SQL_query] # should be type list[str] ?
run_queries(conn, queries = queries)

and run mypy over that (at least inside my complex codebase) I will
get a type mismatch being hinted at.

So far I don't grasp at which point my reasoning above is faulty.

Karsten

--- MBSE BBS v1.0.8.4 (Linux-x86_64)
* Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)

On Sun, 31 Dec 2023 at 03:38, Thomas Passin via Python-list <python-list@python.org> wrote:

I am not very expert in Python type hints. In working up the example
program I just posted, I got an error message from mypy that remarked
that "list" is invariant, and to try Sequence which is "covariant". I
don't know what that means (and I haven't looked into it yet), but when
I changed from list to Sequence as suggested, mypy stopped complaining.

Ah, I think you've hit on the problem there. Consider this:

def add_item(stuff: dict[str: str | int]):
stuff["spam"] = "ham"
stuff["vooom"] = 1_000_000

Is it valid to pass this function a dict[str: str]? No, because it's
going to add an integer into it.

Hopefully that helps explain what's going on a bit.

ChrisA

--- MBSE BBS v1.0.8.4 (Linux-x86_64)
* Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)

On 31/12/23 8:05 am, Chris Angelico wrote:

Ah, I think you've hit on the problem there. Consider this:

def add_item(stuff: dict[str: str | int]):
stuff["spam"] = "ham"
stuff["vooom"] = 1_000_000

Yep, that's it exactly. It's not the union itself that's the problem,
but the fact that there's a *mutable container* containing that type.

--
Greg

--- MBSE BBS v1.0.8.4 (Linux-x86_64)
* Origin: ---:- FTN<->UseNet Gate -:--- (3:633/280.2@fidonet)