os.scandir incorrectly reports symlink as file

[Mabooka]

Bug report

Bug description:

I started rewriting my code to replace os.stat for every file with a smarter / more efficient os.scandir.
Nice. Thank you.
However:

os.scandir() entries report symlinks as FILES, -- which is absolutely incorrect and breaks everything.

To reproduce:

vlad@MacBook:~/tmp$ rm *
vlad@MacBook:~/tmp$ touch foo
vlad@MacBook:~/tmp$ ln -s foo bar
vlad@MacBook:~/tmp$ ls -lart 
total 0
drwxr-xr-x  22 vlad  staff  704 Jan 27 22:04 ..
-rw-r--r--   1 vlad  staff    0 Jan 27 22:08 foo
lrwxr-xr-x   1 vlad  staff    3 Jan 27 22:08 bar -> foo
drwxr-xr-x   4 vlad  staff  128 Jan 27 22:08 .

# so we have one file 'foo' and one symlink 'bar' in the cwd.
# Now: 

vlad@MacBook:~/tmp$ python
Python 3.9.6 (default, May  7 2023, 23:32:44) 
[Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> files = []
>>> entries = os.scandir('.')
>>> for e in entries:
...     if e.is_file(): # is FILE? 
...             print("Found a file:", e.name)
...             files.append(e.name)
... 
Found a file: foo
Found a file: bar
>>> files
['foo', 'bar']

# However, UNIX thinks different:  only one file here:
>>> os.system("find . -type f")
./foo
0
>>> 

As one can see here, 'bar' is treated as a file.
The UNIX find cannot find it with the option '-type f'.
It is NOT a file.

Please fix.

CPython versions tested on:

3.9

Operating systems tested on:

macOS

[Mabooka]

Sorry I messed up the markdown... it's very counterintuitive for me.

[a12k]

This is expected behavior, not a bug. By default, os.scandir will follow symlinks and check if it points to a file, if so, it returns True. This is what you're seeing. Perhaps set follow_symlinks=False when you check if is_file gets you the behavior you're expecting?

You can see the arg here: https://docs.python.org/3/library/os.html#os.DirEntry.is_file

This from the PEP seems relevant: https://peps.python.org/pep-0471/#methods-not-following-symlinks-by-default

[picnixz]

Closing as it's a documented behavior.

[Mabooka]

Thank you for pointing out the "follow_symlinks=False".
I'd like to point out however that this logic is different from the default UNIX find logic, which is not good.
In other words, in my opinion the default should be
is_file(*, follow_symlinks=False)
To this I add, that the semantics of follow_symlinks is to follow directories, not files.

Also, in the documentation, it states that there are no attributes, just methods, so clearly is_file is not as optimized as it could be (considering that in the real world there are many more files in the world than links :)).

I will test it and update here.
I'd like to compare 'follow_symlinks' with my current implementation performance-wise.
So far, I (have to!) use the following logic:

`def files_from_dir(full_path):
try:
entries = os.scandir(full_path)
except Exception as e: # NOT OSError as documented!
print("OOPS: os.scandir(%s): exception '%s'" % (full_path, e))
# PermissionError: [Errno 13] Permission denied: '/Users/vlad/Library/Saved Application State/net.steinberg.eLicenser.installer.SteinbergDownloadAssistant.savedState'
return []

res = []
for e in entries:
    if e.is_file():
        if e.is_symlink():
            continue

        res.append(dict_from_entry(e, full_path))
        
return res

`

[Mabooka]

Sorry again: using 'code' markdown does not work properly.