Can't download directories with spaces with get_r

[Rfromearth]

Hello, I'd like to use the get_r method, but it fails, when the remote directory contains folders named like this one:

1.10 oneword

The name of the downloaded folder is '1' and other folders (e.g. 1.10.10 two words) aren't even downloaded.
Everything works well with Filezilla so I believe that the server config is okay.

example.py

import sftpretty

hostname = 'xx.xx.xx'
port = 22
username = 'ftpuser'
password = 'password'
local_dir = '/home/user/local_dir'
remote_dir = '/home/ftpuser/testfiles'

cnopts = sftpretty.CnOpts(knownhosts=None)
cnopts.hostkeys = None
with sftpretty.Connection(host=hostname,
                          port=port,
                          username=username,
                          password=password,
                          cnopts=cnopts) as sftp:
    print(f'Downloading {remote_dir}')
    sftp.chdir(f'{remote_dir}')
    print(sftp.pwd)
    print(sftp.listdir())
    sftp.get_r(sftp.pwd, f'{local_dir}', preserve_mtime=True, resume=True )
    print(sftp.pwd)

Thank you in advance for looking into my issue.

[Rfromearth]

Found the error. The problematic line(s) are in the method remotetree

__init__.py

try:
  localdir = Path(localdir).expanduser().as_posix()
  remotedir = self.normalize(remotedir)
  for attribute in self.listdir_attr(remotedir):
    if S_ISDIR(attribute.st_mode):
      remote = Path(remotedir).joinpath(
                              attribute.filename).as_posix()
      local = Path(localdir).joinpath(
                         Path(remote).stem).as_posix()
     if remotedir in container.keys():
       container[remotedir].append((remote, local))
     else:
       container[remotedir] = [(remote, local)]
     if recurse:
       self.remotetree(container, remote, local,
                                recurse=recurse)
except Exception as err:
  raise err

When I replace
local = Path(localdir).joinpath(Path(remote).stem).as_posix()
with
local = Path(localdir).joinpath(attribute.filename).as_posix()
get_r works for me.

But I saw a few other occurences of Path().stem which may produce other effects. I think pathnames should never be altered or restricted in 1:1 copy operations,

[byteskeptical]

Thank you for taking the time to look into it further I will test this on my end and get back to you. What os(s) are involved in the transfer in your example? Python version? I agree in general with not altering pathnames but that is also not what is going on with the use of .stem in that function. It is attempting to grab only the immediate parent directory in order to properly build the correct file structure under the user provided local directory for recursive downloads.

You provide /home/user/local_dir. Let's assume the following structure exists within that remote directory you're getting these files from, in this case /home/ftpuser/testfiles.

foo
    |- test.txt
bar
    |- baz
        |- test2.txt

you would not expect to find /home/ftpuser included in the directory structure created under your local destination /home/user/local_dir. You would probably just expect to see the testfiles directory with foo and bar as sub-directories within keeping the rest of the nested structure intact. Clearly there is some limitation being hit by the .stem function's ability to parse the correct ending to some of your directory names.

[Rfromearth]

My local computer and the actual used server are running both debian 12 and the 'local_dir' as user's home directory was just meant as example. The local Python version is 3.13.10.
The next step was the recursive deletion of the remote directory like this:

def delete_r(sftp_conn, remote_path):
    print(f"Starting recursive deletion of '{remote_path}'…")
    _recursive_delete(sftp_conn, remote_path)
    print("Recursive deletion completed")

def _recursive_delete(sftp_conn, remote_path):
    try:
        for item in sftp_conn.listdir_attr(remote_path):
            remote_item_path = remote_path + '/' + item.filename
            if stat.S_ISDIR(item.st_mode):
                print(f'remove directory {remote_item_path}')
                _recursive_delete(sftp_conn, remote_item_path)
                sftp_conn.rmdir(remote_item_path)
            else:
                print(f'remove file {remote_item_path}')
                sftp_conn.remove(remote_item_path)

    except IOError as e:
        print(f"Couldn't process {remote_path}n: {e}")

delete_r(sftp, remote_dir)

It works, while

sftp.put_r(localdir=local_dir, remotedir=remote_dir, preserve_mtime=True, resume=True)

throws directly [Errno 2] No such file

and doesn't copy anything.

[byteskeptical]

Got a fix in #72 if you get a chance to try it out. Working on adding a test for this issue to help prevent regressions, otherwise the code is ready. On a side note if you're looking to do something like a recursive delete, you could reuse the remotetree dictionary to avoid having to duplicate the path building logic.

[Rfromearth]

Thank you for your corrections. Get_r is working now but put_r only creates the last part of the local_dir-string on the remote site - which should't be - but transfers neither files nor directories even if they don't have dots or spaces in their names.

[Rfromearth]

These changes made put_r work for me. Don't mind my german comments. The interesting parts follow after KORREKTUR 1 and 2.
And there is another point: Calling put_r(resume=True) will always fail, so it has be set to False.

    def put_d(self, localdir, remotedir, callback=None, confirm=True,
              preserve_mtime=False, resume=False, workers=None,
              exceptions=None, tries=None, backoff=2, delay=1,
              logger=getLogger(__name__), silent=False):
        '''Copies a local directory's contents to a remotepath'''
        localdir = Path(localdir)

        # KORREKTUR 1: Erstelle das korrekte Zielverzeichnis.
        # put_r übergibt bereits den vollständigen, korrekten Pfad.
        self.mkdir_p(remotedir)

        # Erstelle die Liste der Dateien, die hochgeladen werden sollen.
        # Die Struktur ist: (lokaler_pfad, remote_pfad, ...)
        paths = [
            (
                localpath.as_posix(),  # Der vollständige lokale Pfad zur Datei
                # KORREKTUR 2: Erstelle den korrekten Remote-Pfad relativ zum übergebenen lokalen Verzeichnis.
                Path(remotedir).joinpath(localpath.relative_to(localdir).as_posix()).as_posix(),
                callback, confirm, preserve_mtime, resume, # KORREKTUR 3: Verwende den Funktionsparameter 'resume'
                exceptions, tries, backoff, delay, logger, silent
            )
            for localpath in localdir.iterdir()  # Iteriere über das LOKALE Verzeichnis
            if localpath.is_file()
        ]

        if paths != []:
            thread_prefix = uuid4().hex
            with ThreadPoolExecutor(max_workers=workers,
                                    thread_name_prefix=thread_prefix) as pool:
                logger.debug(f'Thread Prefix: [{thread_prefix}]')
                threads = {
                    pool.submit(
                        self.put, local, remote, callback=callback,
                        confirm=confirm, preserve_mtime=preserve_mtime,
                        resume=False, exceptions=exceptions, tries=tries,
                        backoff=backoff, delay=delay, logger=logger,
                        silent=silent
                    ): local
                    for local, remote, callback, confirm,
                    preserve_mtime, resume, exceptions, tries, backoff,
                    delay, logger, silent in paths
                }

                for future in as_completed(threads):
                    name = threads[future]
                    try:
                        data = future.result()
                    except Exception as err:
                        logger.error(f'Thread [{name}]: [FAILED]')
                        raise err
                    else:
                        logger.info(f'Thread [{name}]: [COMPLETE]')
                        return data
        else:
            logger.info(f'No files found in directory [{localdir}]')

[byteskeptical]

Did you seriously just send me this GPT bullshit?

[Rfromearth]

Opinion is sometimes the killer of knowledge. You're completely mistaken, mate. I'm only one of those Assembler and Turbo-C Programmers from the 80s, who wrote their comments directly in their source files. Documenting anything? Okay, but with Wordstar 2000. I spent many years with debugging programs and do you know what? I hated it.
Now I'm in the comfortable situation not to depend any more on programming. It's just a hobby like solving crosswords.
I assure you that I use ChatGPT (R. Stallman: 'bullshit generators') only for creating porn.I swear.
I don't know everything about Python and even less about git and/or github, because - I mentioned it - I don't depend on it.

One question: When people are against bullshit generators, why don't they leave github?
I really don't understand that. It was absolutely clear that Microsoft would try to monetize everything when they became the owners of github in 2018.

Thank you for your library and have a nice(r) time.

[byteskeptical]

God I hope that was AI as well but it sounds too pathetic even for a robot. So comfortable that your on here wasting people's time about something you claim to know nothing about and treat like a crossword puzzle? But you can't wait for a few days for someone to fix your issue so you gaslight them about using a 'bullshit generator'? Sounds like you do really hate debugging, so leave it to those that don't. You sound too cool for me anyway.