Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev/black #3840

Merged
merged 5 commits into from
Jul 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 3 additions & 3 deletions .dev_scripts/diff_images.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,13 @@ def calc_images_mean_L1(image1_path, image2_path):

def parse_args():
parser = argparse.ArgumentParser()
parser.add_argument('image1_path')
parser.add_argument('image2_path')
parser.add_argument("image1_path")
parser.add_argument("image2_path")
args = parser.parse_args()
return args


if __name__ == '__main__':
if __name__ == "__main__":
args = parse_args()
mean_L1 = calc_images_mean_L1(args.image1_path, args.image2_path)
print(mean_L1)
1 change: 1 addition & 0 deletions .git-blame-ignore-revs
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
b3dccfaeb636599c02effc377cdd8a87d658256c
218b6d0546b990fc449c876fb99f44b50c4daa35
27 changes: 27 additions & 0 deletions .github/workflows/style-checks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: Black # TODO: add isort and flake8 later

on:
pull_request: {}
push:
branches: master
tags: "*"

jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.10'

- name: Install dependencies with pip
run: |
pip install --upgrade pip wheel
pip install .[test]

# - run: isort --check-only .
- run: black --check .
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you're using pre-commit, may I recommend the following action instead:

      - uses: pre-commit/action@v3.0.0

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pretty cool. I did not know this existed. Personally I'm a little torn on using it because there's not much gained by using it and it leaves us dependent on an externally managed action. There's also a slight difference in what is being run in the precommit hooks right now. For the sake of speed, they only run black on what is being committed, while the current GHA is running black on everything.
If others prefer using this action, though, I don't feel super strongly about it.

Copy link

@mikeage mikeage Jul 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a dev here, so please remember that I cannot speak for the project at all :-). I understand what you're saying about external actions, although it is pretty official. Perhaps at least run pre-commit run -a instead of black directly? It reduces the chance that at some point someone might want to change a parameter or something and will need to remember to sync in both places.

IIRC, I think the action it can also be configured to add comments with places that were missed if you pass a token (but possibly not on a PR from a fork). It also includes a git diff after it runs, so instead of just a binary "failed", you can see what it fixed, which can be helpful for new contributors who aren't aware of the coding conventions.

# - run: flake8
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,6 @@ develop-eggs/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
Expand Down
10 changes: 10 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# See https://pre-commit.com/ for usage and config
repos:
- repo: local
hooks:
- id: black
name: black
stages: [commit]
language: system
entry: black
types: [python]
55 changes: 26 additions & 29 deletions installer/lib/installer.py
Original file line number Diff line number Diff line change
Expand Up @@ -141,15 +141,16 @@ def app_venv(self, path: str = None):

# upgrade pip in Python 3.9 environments
if int(platform.python_version_tuple()[1]) == 9:

from plumbum import FG, local

pip = local[get_pip_from_venv(venv_dir)]
pip[ "install", "--upgrade", "pip"] & FG
pip["install", "--upgrade", "pip"] & FG

return venv_dir

def install(self, root: str = "~/invokeai-3", version: str = "latest", yes_to_all=False, find_links: Path = None) -> None:
def install(
self, root: str = "~/invokeai-3", version: str = "latest", yes_to_all=False, find_links: Path = None
) -> None:
"""
Install the InvokeAI application into the given runtime path

Expand All @@ -175,7 +176,7 @@ def install(self, root: str = "~/invokeai-3", version: str = "latest", yes_to_al
self.instance = InvokeAiInstance(runtime=self.dest, venv=self.venv, version=version)

# install dependencies and the InvokeAI application
(extra_index_url,optional_modules) = get_torch_source() if not yes_to_all else (None,None)
(extra_index_url, optional_modules) = get_torch_source() if not yes_to_all else (None, None)
self.instance.install(
extra_index_url,
optional_modules,
Expand All @@ -188,6 +189,7 @@ def install(self, root: str = "~/invokeai-3", version: str = "latest", yes_to_al
# run through the configuration flow
self.instance.configure()


class InvokeAiInstance:
"""
Manages an installed instance of InvokeAI, comprising a virtual environment and a runtime directory.
Expand All @@ -196,7 +198,6 @@ class InvokeAiInstance:
"""

def __init__(self, runtime: Path, venv: Path, version: str) -> None:

self.runtime = runtime
self.venv = venv
self.pip = get_pip_from_venv(venv)
Expand Down Expand Up @@ -312,7 +313,7 @@ def install_app(self, extra_index_url=None, optional_modules=None, find_links=No
"install",
"--require-virtualenv",
"--use-pep517",
str(src)+(optional_modules if optional_modules else ''),
str(src) + (optional_modules if optional_modules else ""),
"--find-links" if find_links is not None else None,
find_links,
"--extra-index-url" if extra_index_url is not None else None,
Expand All @@ -329,15 +330,15 @@ def configure(self):

# set sys.argv to a consistent state
new_argv = [sys.argv[0]]
for i in range(1,len(sys.argv)):
for i in range(1, len(sys.argv)):
el = sys.argv[i]
if el in ['-r','--root']:
if el in ["-r", "--root"]:
new_argv.append(el)
new_argv.append(sys.argv[i+1])
elif el in ['-y','--yes','--yes-to-all']:
new_argv.append(sys.argv[i + 1])
elif el in ["-y", "--yes", "--yes-to-all"]:
new_argv.append(el)
sys.argv = new_argv

import requests # to catch download exceptions
from messages import introduction

Expand All @@ -353,16 +354,16 @@ def configure(self):
invokeai_configure()
succeeded = True
except requests.exceptions.ConnectionError as e:
print(f'\nA network error was encountered during configuration and download: {str(e)}')
print(f"\nA network error was encountered during configuration and download: {str(e)}")
except OSError as e:
print(f'\nAn OS error was encountered during configuration and download: {str(e)}')
print(f"\nAn OS error was encountered during configuration and download: {str(e)}")
except Exception as e:
print(f'\nA problem was encountered during the configuration and download steps: {str(e)}')
print(f"\nA problem was encountered during the configuration and download steps: {str(e)}")
finally:
if not succeeded:
print('To try again, find the "invokeai" directory, run the script "invoke.sh" or "invoke.bat"')
print('and choose option 7 to fix a broken install, optionally followed by option 5 to install models.')
print('Alternatively you can relaunch the installer.')
print("and choose option 7 to fix a broken install, optionally followed by option 5 to install models.")
print("Alternatively you can relaunch the installer.")

def install_user_scripts(self):
"""
Expand All @@ -371,11 +372,11 @@ def install_user_scripts(self):

ext = "bat" if OS == "Windows" else "sh"

#scripts = ['invoke', 'update']
scripts = ['invoke']
# scripts = ['invoke', 'update']
scripts = ["invoke"]

for script in scripts:
src = Path(__file__).parent / '..' / "templates" / f"{script}.{ext}.in"
src = Path(__file__).parent / ".." / "templates" / f"{script}.{ext}.in"
dest = self.runtime / f"{script}.{ext}"
shutil.copy(src, dest)
os.chmod(dest, 0o0755)
Expand Down Expand Up @@ -420,11 +421,7 @@ def set_sys_path(venv_path: Path) -> None:
# filter out any paths in sys.path that may be system- or user-wide
# but leave the temporary bootstrap virtualenv as it contains packages we
# temporarily need at install time
sys.path = list(filter(
lambda p: not p.endswith("-packages")
or p.find(BOOTSTRAP_VENV_PREFIX) != -1,
sys.path
))
sys.path = list(filter(lambda p: not p.endswith("-packages") or p.find(BOOTSTRAP_VENV_PREFIX) != -1, sys.path))

# determine site-packages/lib directory location for the venv
lib = "Lib" if OS == "Windows" else f"lib/python{sys.version_info.major}.{sys.version_info.minor}"
Expand All @@ -433,7 +430,7 @@ def set_sys_path(venv_path: Path) -> None:
sys.path.append(str(Path(venv_path, lib, "site-packages").expanduser().resolve()))


def get_torch_source() -> (Union[str, None],str):
def get_torch_source() -> (Union[str, None], str):
"""
Determine the extra index URL for pip to use for torch installation.
This depends on the OS and the graphics accelerator in use.
Expand Down Expand Up @@ -461,9 +458,9 @@ def get_torch_source() -> (Union[str, None],str):
elif device == "cpu":
url = "https://download.pytorch.org/whl/cpu"

if device == 'cuda':
url = 'https://download.pytorch.org/whl/cu117'
optional_modules = '[xformers]'
if device == "cuda":
url = "https://download.pytorch.org/whl/cu117"
optional_modules = "[xformers]"

# in all other cases, Torch wheels should be coming from PyPi as of Torch 1.13

Expand Down
2 changes: 1 addition & 1 deletion installer/lib/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@
type=Path,
default=None,
)

args = parser.parse_args()

inst = Installer()
Expand Down
21 changes: 14 additions & 7 deletions installer/lib/messages.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,13 +36,15 @@


def welcome():

@group()
def text():
if (platform_specific := _platform_specific_help()) != "":
yield platform_specific
yield ""
yield Text.from_markup("Some of the installation steps take a long time to run. Please be patient. If the script appears to hang for more than 10 minutes, please interrupt with [i]Control-C[/] and retry.", justify="center")
yield Text.from_markup(
"Some of the installation steps take a long time to run. Please be patient. If the script appears to hang for more than 10 minutes, please interrupt with [i]Control-C[/] and retry.",
justify="center",
)

console.rule()
print(
Expand All @@ -58,6 +60,7 @@ def text():
)
console.line()


def confirm_install(dest: Path) -> bool:
if dest.exists():
print(f":exclamation: Directory {dest} already exists :exclamation:")
Expand Down Expand Up @@ -92,7 +95,6 @@ def dest_path(dest=None) -> Path:
dest_confirmed = confirm_install(dest)

while not dest_confirmed:

# if the given destination already exists, the starting point for browsing is its parent directory.
# the user may have made a typo, or otherwise wants to place the root dir next to an existing one.
# if the destination dir does NOT exist, then the user must have changed their mind about the selection.
Expand Down Expand Up @@ -300,15 +302,20 @@ def introduction() -> None:
)
console.line(2)

def _platform_specific_help()->str:

def _platform_specific_help() -> str:
if OS == "Darwin":
text = Text.from_markup("""[b wheat1]macOS Users![/]\n\nPlease be sure you have the [b wheat1]Xcode command-line tools[/] installed before continuing.\nIf not, cancel with [i]Control-C[/] and follow the Xcode install instructions at [deep_sky_blue1]https://www.freecodecamp.org/news/install-xcode-command-line-tools/[/].""")
text = Text.from_markup(
"""[b wheat1]macOS Users![/]\n\nPlease be sure you have the [b wheat1]Xcode command-line tools[/] installed before continuing.\nIf not, cancel with [i]Control-C[/] and follow the Xcode install instructions at [deep_sky_blue1]https://www.freecodecamp.org/news/install-xcode-command-line-tools/[/]."""
)
elif OS == "Windows":
text = Text.from_markup("""[b wheat1]Windows Users![/]\n\nBefore you start, please do the following:
text = Text.from_markup(
"""[b wheat1]Windows Users![/]\n\nBefore you start, please do the following:
1. Double-click on the file [b wheat1]WinLongPathsEnabled.reg[/] in order to
enable long path support on your system.
2. Make sure you have the [b wheat1]Visual C++ core libraries[/] installed. If not, install from
[deep_sky_blue1]https://learn.microsoft.com/en-US/cpp/windows/latest-supported-vc-redist?view=msvc-170[/]""")
[deep_sky_blue1]https://learn.microsoft.com/en-US/cpp/windows/latest-supported-vc-redist?view=msvc-170[/]"""
)
else:
text = ""
return text
8 changes: 2 additions & 6 deletions invokeai/app/api/dependencies.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,9 +78,7 @@ def initialize(config: InvokeAIAppConfig, event_handler_id: int, logger: Logger
image_record_storage = SqliteImageRecordStorage(db_location)
image_file_storage = DiskImageFileStorage(f"{output_folder}/images")
names = SimpleNameService()
latents = ForwardCacheLatentsStorage(
DiskLatentsStorage(f"{output_folder}/latents")
)
latents = ForwardCacheLatentsStorage(DiskLatentsStorage(f"{output_folder}/latents"))

board_record_storage = SqliteBoardRecordStorage(db_location)
board_image_record_storage = SqliteBoardImageRecordStorage(db_location)
Expand Down Expand Up @@ -125,9 +123,7 @@ def initialize(config: InvokeAIAppConfig, event_handler_id: int, logger: Logger
boards=boards,
board_images=board_images,
queue=MemoryInvocationQueue(),
graph_library=SqliteItemStorage[LibraryGraph](
filename=db_location, table_name="graphs"
),
graph_library=SqliteItemStorage[LibraryGraph](filename=db_location, table_name="graphs"),
graph_execution_manager=graph_execution_manager,
processor=DefaultInvocationProcessor(),
configuration=config,
Expand Down