UGithubDownloader + some functions and methods for downloading files and folders from repo or downloading all repo.

examples/github/download.md

@@ -0,0 +1,122 @@
+# GitHub / Download
+
+## Introduction
+
+In `ufpy` there are 3 functions and 1 class for downloading things from public GitHub repositories.
+You can download an entire repository, a folder or multiple folders, and a file or multiple files without
+a GitHub token. You can use the `UGithubDownloader` class for all these actions or the other 3 functions.
+The class is more optimized for multiple requests. If you need to download multiple times, use it. It uses
+an unpacked zip archive for all operations and deletes it at the end. If you don't want to download anything
+multiple times, use the functions instead.
+
+Import functions and class from `ufpy`:
+```python
+import ufpy
+from ufpy import UGithubDownloader, download_file, download_folder, download_repo
+```
+
+> [!CAUTION]
+> All repositories you want to download from must be public
+
+## Open `UGithubDownloader` class and use it
+
+To open this class, you should use the `with` operator as you do with files and other resources:
+```python
+with UGithubDownloader("honey-team/ufpy", "C:/Ufpy-Test", "0.1") as gd:
+    # First argument - "repository owner"/"repository name"
+    # Second - Base download path (all methods use download paths relative to the base download path, similar to how it works in the command line.
+    # For example: base path: C:\; cd ufpy -> final path: C:\ufpy.) (default is the current working directory)
+    # Third argument: Branch or tag name (default is "main" (not master!))
+    gd.download_repo() # In C:/Ufpy-Test will appear all files from 0.1 tag in this repository.
+    gd.download_repo("ufpy-0.1") # In C:/Ufpy-Test/ufpy-0.1 will appear all files from 0.1 tag in this repository
+```
+
+## Download file(s)
+
+You can use `download_file()` function, `ufpy.github.download.file()` function
+(they're the same, but with different names) and `UGithubDownloader.download_file()` method.
+
+> [!NOTE]
+> You can use any iterable of strings in `download_file()` function for downloading several files.
+> In `UGithubDownloader`, there is a `download_files()` method.
+
+One file:
+```python
+download_file("honey-team/ufpy", "README.md", "C:/Users/<name>/ufpy-tests")
+# copy README.md from the main branch to the C:/Users/<name>/ufpy-tests directory
+
+with UGithubDownloader("honey-team/ufpy", "C:/Users/<name>/ufpy-tests") as gd:
+    gd.download_file("README.md") # Same
+```
+
+After changing `<name>` to your username and running this code you'll get this:
+![Download one file](.assets/download1.png)
+
+Two files:
+```python
+download_file("honey-team/ufpy", ["README.md", "mkdocs.yml"], "C:/Users/<name>/ufpy-tests")
+# copy README.md and mkdocs.yml from main branch in C:/Users/<name>/ufpy-tests directory
+
+with UGithubDownloader("honey-team/ufpy", "C:/Users/<name>/ufpy-tests") as gd:
+    gd.download_files(["README.md", "mkdocs.yml"]) # Same
+```
+
+After changing `<name>` to your username and running this code you'll get this:
+![Download two files](.assets/download2.png)
+
+## Download folder(s)
+
+You can use `download_folder()` function, `ufpy.github.download.folder()` function
+(they're the same, but with different names) and `UGithubDownloader.download_folder()` method.
+
+> [!NOTE]
+> You can use any iterable of strings in `download_folder()` function for downloading several folders.
+> In `UGithubDownloader` there are `download_folders()` method.
+
+One folder:
+```python
+download_folder("honey-team/ufpy", "examples", "C:/Users/<name>/ufpy-tests")
+# create the C:/Users/<name>/ufpy-tests/examples folder
+# and copy the contents of origin/examples from the main branch into this folder
+
+with UGithubDownloader("honey-team/ufpy", "C:/Users/<name>/ufpy-tests") as gd:
+    gd.download_folder("examples") # Same
+```
+
+After changing `<name>` to your username and running this code you'll get this:
+![Download one folder](.assets/download3.png)
+
+Two folders:
+```python
+download_folder("honey-team/ufpy", ["examples", ".github"], "C:/Users/<name>/ufpy-tests")
+# create C:/Users/<name>/ufpy-tests/examples and C:/Users/<name>/ufpy-tests/.github folders
+# and copy origin/examples contents and origin/.github contents from main branch in these folders
+
+with UGithubDownloader("honey-team/ufpy", "C:/Users/<name>/ufpy-tests") as gd:
+    gd.download_folders(["examples", ".github"]) # Same
+```
+
+After changing `<name>` to your username and running this code you'll get this:
+![Download two folders](.assets/download4.png)
+
+## Download all repository
+
+You can use `download_repo()` function, `ufpy.github.download.repo()` function
+(they're the same, but with different names) and `UGithubDownloader.download_repo()` method:
+```python
+download_repo("honey-team/ufpy", "C:/Users/<name>/ufpy-tests")
+# copy all files and folders from the repository along with their contents
+# from the main branch to the C:/Users/<name>/ufpy-tests directory.
+
+with UGithubDownloader("honey-team/ufpy", "C:/Users/<name>/ufpy-tests") as gd:
+    gd.download_repo() # Same
+```
+
+After changing `<name>` to your username and running this code you'll get this:
+
+> [!NOTE]
+> The repository code shown reflects the state before merging pull request
+> [#37](https://github.com/honey-team/ufpy/pull/37).
+> When this pull request was merged, repository was changed.
+
+![Download all repository](.assets/download5.png)

requirements.txt

@@ -0,0 +1 @@
+requests>=2.31.0

setup.py

@@ -6,14 +6,17 @@
     long_description = mdf.read()
 
 install_requires = [
-
+    'requests>=2.31.0',
 ]
 
 organization_name = 'honey-team'
 author, author_email = 'bleudev', '[email protected]'
 project_name = 'ufpy'
 github_url = f'https://github.com/{organization_name}/{project_name}'
 
+def __package(name: str) -> str:
+    return f'{project_name}.{name}'
+
 setup(
     name=project_name,
     version=__version__,
@@ -24,7 +27,11 @@
     long_description=long_description,
     long_description_content_type='text/markdown',
     url=github_url,
-    packages=[project_name, f'{project_name}.typ'],
+    packages=[
+        project_name,
+        __package('typ'),
+        __package('github'),
+    ],
     classifiers=[
         'Programming Language :: Python :: 3',
         'Programming Language :: Python :: 3.12',

ufpy/__init__.py

@@ -1,12 +1,19 @@
 __version__ = '0.1.2'
 
-# Typing package
-from ufpy import typ
 from ufpy.cmp import *
 from ufpy.math_op import *
-from ufpy.path_tools import *
-from ufpy.typ.protocols import *
-from ufpy.typ.type_alias import *
 from ufpy.udict import *
 from ufpy.ustack import *
 from ufpy.utils import *
+
+# Typing package
+__typ_version__ = '0.1'
+from ufpy.typ import *
+
+# Path package
+__path_version__ = '0.1'
+from ufpy.path import *
+
+# GitHub package
+__github_version__ = '0.1'
+from ufpy.github import *

ufpy/github/__init__.py

@@ -0,0 +1,6 @@
+# Download
+import ufpy.github.download
+from ufpy.github.download import UGithubDownloader
+from ufpy.github.download import file as download_file
+from ufpy.github.download import folder as download_folder
+from ufpy.github.download import repo as download_repo

ufpy/github/download.py

@@ -0,0 +1,130 @@
+import io
+import os
+from shutil import copy, copytree, rmtree
+from tempfile import gettempdir
+from typing import Iterable
+from zipfile import ZipFile
+
+from requests import get
+
+from ufpy.path import UOpen
+
+__all__ = (
+    'file',
+    'folder',
+    'repo',
+    'UGithubDownloader',
+)
+
+def file(repo: str, file_path: str | list[str], download_path: str, branch_name: str = 'main'):
+    with UGithubDownloader(repo, download_path, branch_name) as gd:
+        if isinstance(file_path, str):
+            gd.download_file(file_path)
+        else:
+            gd.download_files(file_path)
+
+def folder(repo: str, folder_path: str | list[str], download_path: str, branch_name: str = 'main'):
+    with UGithubDownloader(repo, download_path, branch_name) as gd:
+        if isinstance(folder_path, str):
+            gd.download_folder(folder_path)
+        else:
+            gd.download_folders(folder_path)
+
+def repo(repo: str, download_path: str, branch_name: str = 'main'):
+    with UGithubDownloader(repo, download_path, branch_name) as gd:
+        gd.download_repo()
+
+
+def format_paths(*paths: str | list[str]) -> str | list[str] | list[list[str]]:
+    new_paths = []
+    for path in paths:
+        if isinstance(path, list):
+            path = format_paths(*path)
+        else:
+            path = path.replace('\\', '/')
+
+            if path.startswith('/'):
+                path = path[1:]
+            if path.endswith('/'):
+                path = path[:-1]
+
+        new_paths.append(path)
+    return new_paths[0] if len(new_paths) <= 1 else new_paths
+
+
+CWD = os.getcwd()
+
+class UGithubDownloader:
+    def __init__(self, repo: str, base_download_path: str = CWD, branch_name: str = 'main'):
+        self.__repo = repo
+        self.__base_download_path = format_paths(base_download_path)
+        self.__branch = branch_name
+
+    def __enter__(self):
+        url = f'https://github.com/{self.__repo}/archive/{self.__branch}.zip'
+        r = get(url, timeout=10)
+
+        if not r.ok:
+            r.raise_for_status()
+
+        self.__zip = ZipFile(io.BytesIO(r.content))
+
+        temp_dir = format_paths(gettempdir())
+        self.__zip.extractall(temp_dir)
+        repo_name = self.__repo.split('/')[-1]
+        self.__repo_path = f'{temp_dir}/{repo_name}-{self.__branch}'
+        return self
+
+    def __exit__(self, exc_type, exc_val, exc_tb):
+        self.__zip.close()
+        if os.path.exists(self.__repo_path):
+            rmtree(self.__repo_path)
+
+    def download_file(self, file_path: str, download_path: str = ''):
+        file_path, download_path = format_paths(file_path, download_path)
+        download_path = f'{self.__base_download_path}/{download_path}'
+
+        url = f'https://raw.githubusercontent.com/{self.__repo}/{self.__branch}/{file_path}'
+        r = get(url)
+
+        if not r.ok:
+            r.raise_for_status()
+
+        path = f'{download_path}/{file_path}'
+
+        with UOpen(path, 'w+') as f:
+            f.write(r.text)
+
+    def download_files(self, file_paths: Iterable[str], download_path: str = ''):
+        file_paths, download_path = format_paths(list(file_paths), download_path)
+        for file_path in file_paths:
+            self.download_file(file_path, download_path)
+
+    def download_folder(self, folder_path: str, download_path: str = ''):
+        download_path = format_paths(download_path)
+        download_path = f'{self.__base_download_path}/{download_path}'
+
+        src, dst = f'{self.__repo_path}/{folder_path}', f'{download_path}/{folder_path}'
+        if os.path.exists(dst):
+            rmtree(dst)
+        copytree(src, dst)
+
+    def download_folders(self, folder_paths: Iterable[str], download_path: str = ''):
+        folder_paths, download_path = format_paths(list(folder_paths), download_path)
+        for folder_path in folder_paths:
+            self.download_folder(folder_path, download_path)
+
+    def download_repo(self, download_path: str = ''):
+        download_path = format_paths(download_path)
+        download_path = f'{self.__base_download_path}/{download_path}'
+
+        for filename in os.listdir(self.__repo_path):
+            src, dst = f'{self.__repo_path}/{filename}', f'{download_path}/{filename}'
+            if os.path.isdir(src):
+                if os.path.exists(dst):
+                    rmtree(dst)
+                copytree(src, dst)
+            else:
+                if os.path.exists(dst):
+                    os.remove(dst)
+                copy(src, dst)

ufpy/path/__init__.py

@@ -0,0 +1,2 @@
+from ufpy.path.files import *
+from ufpy.path.tools import *

ufpy/path/files.py

@@ -0,0 +1,38 @@
+import os
+from typing import AnyStr, Iterable
+
+
+__all__ = (
+    'UOpen',
+)
+
+class UOpen:
+    def __init__(
+            self, filepath: str, mode: str = "r", encoding: str = 'utf-8'
+    ):
+        self.__path = filepath
+        self.__mode = mode
+        self.__encoding = encoding
+
+    def __enter__(self):
+        directory = '/'.join(self.__path.split('/')[:-1]) or '\\'.join(self.__path.split('\\')[:-1])
+
+        os.makedirs(directory, exist_ok=True)
+
+        self.__f = open(file=self.__path, mode=self.__mode, encoding=self.__encoding)
+        return self
+
+    def __exit__(self, exc_type, exc_val, exc_tb):
+        self.__f.close()
+
+    def write(self, data: AnyStr):
+        self.__f.write(data)
+
+    def writelines(self, lines: Iterable[AnyStr]):
+        self.__f.writelines(lines)
+
+    def read(self, n: int = -1) -> AnyStr:
+        return self.__f.read(n)
+
+    def readlines(self, hint: int = -1) -> list[AnyStr]:
+        return self.__f.readlines(hint)