Best practice linking Pythonista with Working Copy #16
-
|
Can I add my “Welcome Back “ appreciation to Ole and for the rebirth of this forum. Pythonista had rekindled my love for programming. I have been developing Pythonista games for the last year (https://github.com/mmccthomas/Pythonista_games), and have been struggling with the interaction of Pythonista and Working Copy. There are 4 possible locations for my folder ‘Pythonista_games’
Any insights or solutions are warmly welcomed. |
Beta Was this translation helpful? Give feedback.
Replies: 6 comments 6 replies
-
|
Not touched in a long time... https://github.com/cclauss/Pythonista-and-Working-Copy |
Beta Was this translation helpful? Give feedback.
-
|
Thank you Christian, and thank you for all the work that you have done for Pythonista over the past few years since the community board went down. I did look at your code, but I couldn’t quite understand whether it assumed that development code was in the Pythonista folder or elsewhere. Currently I have to individually throw single files “over the wall” to and from Working Copy using share sheets and asset scripts. I have to keep track of changed files to do this. There Must be a Better Way! The issue with running Pythonista code directly from the Working Copy folder or via linked Files folder seems to be related to sys.path permissions issues, which I have not been able to resolve. Currently I run script below in each modules to traverse my folder structure to get subfolders. I think it is very hacky! So to be clear what the intrinsic problem is: Any insights ? Best Regards, import os
import sys
# find the path to current file
# back up until Pythonista_games
# add subdirectories to sys path
def add_paths(filename):
current = os.path.dirname(os.path.realpath(filename))
path_item = current
while not path_item.endswith('Pythonista_games') and not path_item.endswith('Documents'):
path_item = os.path.dirname(path_item)
paths = []
try:
paths.append(path_item)
for f in ['gui', 'Board_Games', 'Word_Games', 'Card_Games']:
paths.append(path_item + '/' + f)
[sys.path.append(d) for d in paths]
return paths
#dirs = [d.path for d in os.scandir(path_item) if d.is_dir()]
except (Exception) as e:
print(e)
if __name__ == '__main__':
pass
#add_paths(__file__) |
Beta Was this translation helpful? Give feedback.
-
|
These are the options I have tried:
I do not know if this is the "Best Practice", but it works for me. I would not put the repository in an "iCloud Drive" folder on the iPad, since the local copy may get removed if the disk space gets tight. P.S.: You could put the "base_path.py" into the same folder as the file that wants to import it. You could also use "os.getcwd()" instead of os.path.dirname(file). |
Beta Was this translation helpful? Give feedback.
-
from pathlib import Path
here = Path(__file__).parent |
Beta Was this translation helpful? Give feedback.
-
|
I think Chris' original problem had to do with access rights of Python scripts that were not in Pythonista's Document folder.
I once ran into a problem when I tried to define a "Shortcut" to a Python script that was not stored in Pythonista's "This iPad" Folder. My solution was a helper script that takes the full path of the Python script as the first command line argument, changes the working directory, and executes the Python script that is located outside of Pythonista's "This iPad" document folder. Here is the "run_script.py" code: import importlib.util
import sys
import os
modulefilepath=sys.argv[1]
moduledir,modulefile=os.path.split(modulefilepath)
modulename='__main__'
spec = importlib.util.spec_from_file_location(modulename, modulefilepath)
module = importlib.util.module_from_spec(spec)
sys.modules[modulename] = module
os.chdir(moduledir)
sys.argv=sys.argv[1:]
spec.loader.exec_module(module)This script needs to be in Pythonista's "This iPad" folder so "Shortcuts" can find it. The "Shortcuts" definition would then look something like this: |>| Run run_script.py with /private/var/mobile/Containers/Data/ Application/77881549-3FA6-4E4B-803F-D53B172FC865/ Here is the script I use to copy the full path to the folder where the Python file I want to run is located: import os
import clipboard
clipboard.set(os.getcwd())
print(f'path:"{clipboard.get()}"\n(copied to clipboard)')Pythonista needs to have access rights to the folder that contains the script. I found that using "EXTERNAL FILES> Open...> Folder..." im Pythonista does the trick. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks Richard, Christian for you insights. I think I’ll clear down the IPad and do the link as you suggest. I’m so pleased that a working discussion forum is alive again! |
Beta Was this translation helpful? Give feedback.
-
|
After much experimentation and your advice think I have the following insights. Please forgive me if statements are contentious. I am also happy to be proved wrong!
My solution:
To achieve this I constructed a GUI to display differences between the local folder and cloud (and hence repository) and permit copies in both directions. It also displays the folder and file trees side-by-side for both locations. This GUI replaces the Share Sheet operations for each changed file that were performed previously. I hope that this might benefit others researching the same question. I have scoured the archived forum to answer the same questions. Very glad to discussion further. Chris Thomas # compare files in local Pythonista files and cloud/Pythonista/Pythonista_games
# create a list of Paths for each folder
# using Paths means that position of item does not need to be same in each
# Only compare file contents rather than datetime
# if file does not match, show differences
#
# open dialog to allow difference inspection and choice of which to keep
# Included new files created since last modified date in cloud
#
# must get path or depth, not just name, else matches to other named files
# find text above to get parent?
import filecmp
import os
from itertools import islice
from pathlib import Path
import difflib
import shutil
from itertools import zip_longest
import ui
from objc_util import NSRange, on_main_thread, ObjCClass, ObjCInstance, c_void_p, c
from datetime import datetime
class CompareCloud():
def __init__(self, source):
self.debug = False
self.cloud_base = '/private/var/mobile/Library/Mobile Documents/iCloud~com~omz-software~Pythonista3/Documents/'
self.source = source
self.dest = self.cloud_base + self.source
self.index = 0
self.spacing = 50
self.use_colour = True
def latest_modified(self, which=None):
# get latest modified file and datetime in either cloud or local
if which is None:
which = self.dest
dir_which = Path(which).glob('**/*')
filetimes = {path: datetime.fromtimestamp(os.stat(path).st_mtime)
for path in dir_which}
latest = max(filetimes, key=filetimes.get)
latest_time = filetimes[latest]
return latest, latest_time
def get_new_files(self):
# return the path and datetime of any files created
# after latest cloud file
dir_src = Path(self.source).glob('**/*')
latest_file, latest_dest_time = self.latest_modified(self.dest)
new_files = {p: datetime.fromtimestamp(os.stat(p).st_birthtime)
for p in dir_src
if datetime.fromtimestamp(os.stat(p).st_birthtime) > latest_dest_time}
return new_files
def show_diffs(self, file1, file2):
"""get differences between 2 files """
def file_mtime(path):
# file modified time
t = datetime.fromtimestamp(os.stat(path).st_mtime)
return t.strftime('%a %d %b %Y, %I:%M%p')
try:
with open(file1, 'r', encoding='utf-8') as hosts0:
with open(file2, 'r', encoding='utf-8') as hosts1:
diff = difflib.context_diff(hosts0.readlines(),
hosts1.readlines(),
fromfile='local',
tofile='cloud',
fromfiledate=file_mtime(file1),
tofiledate=file_mtime(file2))
"""
diff = difflib.HtmlDiff().make_table(
hosts0.readlines(),
hosts1.readlines(),
fromdesc='local',
todesc='cloud',
context=True, numlines=5)
"""
except UnicodeDecodeError:
diff = ['unicode error in file']
return diff
def tree(self,
dir_path: Path,
level: int = -1,
limit_to_directories: bool = False,
length_limit: int = 1000,
no_print=False,
exclude=[]):
"""Given a directory Path object print a visual tree structure"""
space = ' '
branch = '│ '
tee = '├── '
last = '└── '
dir_path = Path(dir_path) # accept string coerceable to Path
self.files = 0
self.directories = 0
def inner(dir_path: Path, prefix: str = '', level=-1):
if not level:
return # 0, stop iterating
if limit_to_directories:
contents = [d for d in dir_path.iterdir() if d.is_dir()]
else:
contents = list(dir_path.iterdir())
pointers = [tee] * (len(contents) - 1) + [last]
for pointer, path in zip(pointers, contents):
if path.is_dir() and path.name in exclude:
continue
if path.is_dir():
yield prefix + pointer + path.name
self.directories += 1
extension = branch if pointer == tee else space
yield from inner(path,
prefix=prefix + extension,
level=level - 1)
elif not limit_to_directories:
yield prefix + pointer + path.name
self.files += 1
iterator = inner(dir_path, level=level)
if no_print:
return iterator
else:
print(dir_path.name)
for line in islice(iterator, length_limit):
print(line)
if next(iterator, None):
print(f'... length_limit, {length_limit}, reached, counted:')
print(f'\n{self.directories} directories' +
(f', {self.files} files' if self.files else ''))
def not_excluded(self, f):
# decide if file should be included in comparison
if f.name.split('.')[-1] in [
'npy', 'npz', 'pkl', 'json', 'gitignore', 'git'
]:
return False
if '.git' in f.parts:
return False
return True
def compare(self):
"""compare two directory trees
use Path objects to allow matching
"""
dir_src = Path(self.source).glob('**/*')
dir_dest = list(Path(self.dest).glob('**/*'))
# make cloud version have same path
dir_dest_rel = [p.relative_to(self.cloud_base) for p in dir_dest]
self.new_files = self.get_new_files()
# iterate paths
result_dict = {}
for src in dir_src:
if src.is_file() and self.not_excluded(src):
if src in self.new_files:
with open(src, 'r', encoding='utf-8') as f:
content = f.read()
result_dict[src] = content
elif src in dir_dest_rel:
dest = dir_dest[dir_dest_rel.index(src)]
if filecmp.cmp(src, dest, shallow=False):
pass
# print('match')
else:
self.diffs = self.show_diffs(src, dest)
result_dict[src] = [list(self.diffs)]
if self.debug:
print('#' * 78)
print('found', src)
print('no match')
[print(line) for line in self.diffs]
return result_dict
@on_main_thread
def init_colours(self):
UIColor = ObjCClass('UIColor')
self.UIfont = ObjCClass('UIFont').fontWithName_size_(*self.t.font)
self.tvo = ObjCInstance(self.t)
self.tvo.setAllowsEditingTextAttributes_(True)
self.stro = ObjCClass(
'NSMutableAttributedString').alloc().initWithString_(self.t.text)
self.stro.addAttribute_value_range_(
ObjCInstance(c_void_p.in_dll(c, 'NSFontAttributeName')),
self.UIfont, NSRange(0, len(self.t.text)))
self.colors = {
'red': UIColor.redColor(),
'green': UIColor.greenColor(),
'blue': UIColor.blueColor(),
'cyan': UIColor.cyanColor(),
'magenta': UIColor.magentaColor(),
'black': UIColor.blackColor(),
'yellow': UIColor.yellowColor()
}
def setup_views(self):
""" Create GUI """
def image_scale(name, wi=100, string=''):
# bigger buttons
img = ui.Image.named(name)
w,h = img.size
hi = wi*h/w
with ui.ImageContext(wi,hi) as ctx:
img.draw(0,0,wi,hi)
ui.draw_string(string, rect=(0,4*hi/5,wi, hi), color='black', font=('Menlo',12))
ui_resize = ctx.get_image()
return ui_resize
rb = ui.ButtonItem(# title='Local->Cloud',
image=image_scale('iob:ios7_cloud_upload_32', 72),
enabled=True,
tint_color='darkred',
action=self.to_cloud)
rb1 = ui.ButtonItem(# title='Cloud->Local',
image=image_scale('iob:ios7_cloud_download_32', 72),
enabled=True,
tint_color='darkred',
action=self.to_local)
rb2 = ui.ButtonItem(image=image_scale('iob:ios7_arrow_forward_32', 64), enabled=True, action=self.skip)
rb3 = ui.ButtonItem(image=image_scale('iob:ios7_arrow_back_32', 64), enabled=True, action=self.back)
lb4 = ui.ButtonItem(image=ui.Image.named('iob:refresh_32'), enabled=True, action=self.refresh, tint_color='black')
lb5 = ui.ButtonItem(image=ui.Image.named('emj:Palm_Tree'), enabled=True, action=self.trees, tint_color='black')
self.w, self.h = ui.get_screen_size()
self.main = ui.View(frame=(0, 0, self.w, self.h))
self.main.left_button_items = [lb4, lb5]
self.main.right_button_items = [rb2, rb3, rb, rb1]
self.main.content_size = (self.w, self.h)
# List to select post
self.t = ui.TextView(frame=(0, 0, *self.main.content_size),
text_color='black',
bordered=True,
border_width=2,
font=('Menlo', 20))
# self.wv = ui.WebView(frame=(0, 0, *self.main.content_size),
# bordered=True,
# border_width=2)
# self.wv.load_html(self.diffs)
# set up objc instance
self.t.delegate = self
self.main.add_subview(self.t)
# self.main.add_subview(self.wv)
# self.wv.bring_to_front()
# calculate number of characters across textview
# only true for monospaced fonts
w, h = ui.measure_string('#####', font=self.t.font)
self.chars = int(5 * self.w / w)
self.spacing = self.chars // 2
self.main.present('sheet')
def to_local(self, sender):
# overwrite local file with cloud
if self.items:
local = self.items[self.index]
cloud = Path(self.cloud_base + str(local))
shutil.copyfile(cloud, local)
self.refresh(None)
def to_cloud(self, sender):
# overwrite cloud file with local
if self.items:
local = self.items[self.index]
cloud = Path(self.cloud_base + str(local))
shutil.copyfile(local, cloud)
self.refresh(None)
def skip(self, sender):
if self.index < len(self.items) - 1:
self.index += 1
self.display()
def back(self, sender):
if self.index > 0:
self.index -= 1
self.display()
def refresh(self, sender):
self.t.text = ''
self.result_dict = self.compare()
self.items = list(self.result_dict.keys())
self.index = 0
self.display()
def trees(self, sender):
# get trees for local folder and its corresponding cloud version
length_limit = 10000
s = self.spacing
iterator_dest = self.tree(self.dest,
length_limit=length_limit,
no_print=True,
exclude=['.git'])
iterator_src = self.tree(self.source,
length_limit=length_limit,
no_print=True,
exclude=['.git'])
lines_dest = [line for line in islice(iterator_dest, length_limit)]
header_dest = f'{self.directories} directories' + (
f', {self.files} files' if self.files else '')
self.directories = self.files = 0
lines_src = [line for line in islice(iterator_src, length_limit)]
header_src = f'{self.directories} directories' + (
f', {self.files} files' if self.files else '')
tmod_d = 'Last mod: ' + self.latest_modified(self.dest)[1].strftime('%a %d %b %Y, %I:%M%p')
tmod_s = 'Last mod: ' + self.latest_modified(self.source)[1].strftime('%a %d %b %Y, %I:%M%p')
text = f'{self.source:<{s}} Cloud\n'
text += f'{header_src:<{s}} {header_dest}\n'
text += f'{tmod_s:<{s}} {tmod_d:>}\n'
for line_src, line_dest in zip_longest(lines_src,
lines_dest,
fillvalue=''):
# format the trees to fill the textview
# make all lines spacing length
line_src = f'{line_src[:s]:<{s}}'
line_dest = f'{line_dest[:s]}'
text += f'{line_src} {line_dest} \n'
self.t.text = text
if self.use_colour:
self.colour_tree_items()
def display(self):
if self.items:
text = self.result_dict[self.items[self.index]]
if self.items[self.index] in self.new_files:
self.t.text = text
self.main.name = f'{self.index+1}/{len(self.items)} {self.items[self.index]} -- New File'
else:
self.t.text = ''.join(text[0])
self.main.name = f'{self.index+1}/{len(self.items)} {self.items[self.index]} -- Existing File'
if self.use_colour:
self.colour_display()
else:
self.main.name = 'No Differences'.upper()
@on_main_thread
def colour_tree_items(self):
"""
This code is to add colour to modified files in tree
New files are green, modified files are cyan
"""
self.init_colours()
colors = [self.colors['cyan'], self.colors['green']]
lines = self.t.text.split('\n')
for index, line in enumerate(lines):
for item in self.result_dict:
if item.name in line:
for column in range(2): # allow for item.name on same line
try:
pos = line.index(item.name, column * self.spacing)
if self.get_parent(lines, index, item, column):
start = pos + len('\n'.join(
lines[:index])) # length of all lines so far
end = start + len(item.name) + 1
color = colors[int(item in self.new_files)]
self.stro.addAttribute_value_range_(
'NSBackgroundColor', color,
NSRange(start, end - start))
self.tvo.setAttributedText_(self.stro)
except ValueError:
continue
def get_parent(self, lines, index, item, column=0):
"""
Check if item.name is under correct folder
as filenames can be repeated, e.g. __init__.py
lines is all lines in textview
index is current line number containing item
column (0/1) will search from start or mid line
"""
pos = lines[index].index(item.name, column * self.spacing)
depth = (pos % self.spacing) // 4
parent_name = item.parts[-2]
# search up to find parent
i = index
while i >= 0:
# only search in column
if pos >= self.spacing:
line = lines[i][self.spacing:]
else:
line = lines[i][:self.spacing]
# find position of first alphanumeric character
pos_line = line.find(next(filter(str.isalpha, line)))
if pos_line == -1: # not found
i -= 1
continue
parent_depth = (pos_line % self.spacing) // 4
if parent_depth == depth - 1: # corrent depth
return parent_name == line[pos_line:].strip()
i -= 1
return False
@on_main_thread
# apparently this must be called on main thread for textview
def colour_display(self):
"""
This code is to add colour to text differences
find initial string then
find start text
then find all lines beginning '!'
sounds like a state machine to me!
"""
self.init_colours()
start_texts = ['*** ', '---']
colors = [self.colors['cyan'], self.colors['green']]
init_str = '*' * 15
lines = self.t.text.split('\n')
started = False
found_start_text = False
color = colors[0]
for index, line in enumerate(lines):
if line.startswith(init_str):
started = True
elif line.startswith('!'):
if found_start_text:
start = len('\n'.join(
lines[:index])) # length of all lines so far
end = start + len(lines[index]) + 1
self.stro.addAttribute_value_range_(
'NSBackgroundColor', color,
NSRange(start, end - start))
self.tvo.setAttributedText_(self.stro)
elif line.startswith(start_texts[0]):
if started:
found_start_text = True
color = colors[0]
elif line.startswith(start_texts[1]):
if started:
found_start_text = True
color = colors[1]
def run(self):
if self.debug:
self.tree(self.source)
self.get_new_files()
self.result_dict = self.compare()
self.setup_views()
if self.debug:
print(self.result_dict)
self.items = list(self.result_dict.keys())
self.display()
if __name__ == '__main__':
CompareCloud(source='Pythonista_games').run()
|
Beta Was this translation helpful? Give feedback.
-
|
Hi Chris, I prefer folders in "On My iPad" over "iCloud", because the iCloud folder synchronization is not very predictable. I think Pythonista already has full access to its own document folder "On My iPad/Pythonista 3", even if it is not listed under "External Files". You should have no problems with access restrictions if you put your repository folders inside the "On My iPad/Pythonista 3" folder. Then, may be, you could check if you have the "Link external repository" in "Working Copy" enabled. I think it was introduced in 2020, and you may need to upgrade. You can check this in "Working Copy" under "Settings > Pro capabilities unlocked > Features". Regards, P.S.: Do you have the "On My iPad/Pythonista 3" folder at all? May be you still have to create it. Here is how it looks on my iPad: The "Documents" are the default document folders of other apps (e.g. "Textastic"). You would have to look inside to see which is which. |
Beta Was this translation helpful? Give feedback.
-
|
Hi Richard, That’s interesting, because On My IPad/Pythonista 3 appears as a folder in Pythonista/External Files, but Pythonista 3 doesn’t appear in Working Copy/Link Repository selection but it doesn’t for you? I understand your comment about she Cloud. I have experienced issues when running Pythonista on other devices. If a Cloud module is open when Pythonista is started, the Cloud version can be over written. I do suspect that my folder structure is the root cause of many of my issues, since there is no alternative to sys.path modification to get the imports correct. Thanks for your interest Richard. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, "On My iPad/Pythonista 3" does show up when I try to link a cloned repository: I just cloned your Python_games repository, and then linked it to a folder "Pythonista_games" inside of "On My iPad/Pythonista 3". Since the folder "Pythonista_games" did not exist, I had to create it first, and rename it. It is important that you open the folder before you click "open". The newly created folder should be empty at this point. |
Beta Was this translation helpful? Give feedback.
-
|
You Sir are a Genius!!!! I have tested any of my games on an iPad mini, but in “most” cases there is a fairly obvious size selection function. Thanks for engaging, Richard.I really appreciate it. |
Beta Was this translation helpful? Give feedback.
-
|
I will close this question as Answered, as Richard’s suggestions have worked. |
Beta Was this translation helpful? Give feedback.
I will close this question as Answered, as Richard’s suggestions have worked.
All program work identically if they are moved to the On My iPad folder, and Working Copy can be correctly linked.
Thank you!