bump macos to m1

[t-vi]

some discussion on #1724

seems to run into segfaults. obviously, if anyone with a macbook or so could take over sorting things out, it would be supergood.

[Andrei-Aksionov]

Tried to run API tests locally on M3 chip and had no issues.
Unfortunately, I have no time to debug it.
@rasbt all hope is on you.

P.S. Here is an interesting project: https://github.com/mxschmitt/action-tmate
Haven't tried it, but in theory it should allow you to connect to a runner and check what's what.

[t-vi]

So by commenting out the LLM imports, it makes the segfaults go away, but isn't a realistic option.
Also, tons of failing tests.

[rasbt]

It's a bit of a weird machine. The memory issues aside, I am also seeing

FAILED tests/test_convert_lit_checkpoint.py::test_against_original_gemma_2[device0-dtype0-gemma-2-27b] - AssertionError: Tensor-likes are not close!
Mismatched elements: 305 / 5120000 (0.0%)
Greatest absolute difference: 1.7881393432617188e-05 at index (0, 13, 73469) (up to 1e-05 allowed)

It works perfectly fine locally on both my Macs on M1 and M3. I am also using macOS 14.

[t-vi]

Thank you for looking into this. I'd not worry much about that. This can happen from different rng seeds, test order etc. Probably OK to just increase the tolerance to 3e-5 or so. Am 16. September 2024 19:42:14 MESZ schrieb Sebastian Raschka ***@***.***>:

…

It's a bit of a weird machine. The memory issues aside, I am also seeing > FAILED tests/test_convert_lit_checkpoint.py::test_against_original_gemma_2[device0-dtype0-gemma-2-27b] - AssertionError: Tensor-likes are not close! Mismatched elements: 305 / 5120000 (0.0%) Greatest absolute difference: 1.7881393432617188e-05 at index (0, 13, 73469) (up to 1e-05 allowed) It works perfectly fine locally on both my Macs on M1 and M3. I am also using macOS 14. -- Reply to this email directly or view it on GitHub: #1725 (comment) You are receiving this because you authored the thread. Message ID: ***@***.***>

[rasbt]

I was able to isolate it, it's this one here that segfaults on the CI:

def test_llm_load_random_init(tmp_path):
     download_from_hub(repo_id="EleutherAI/pythia-14m", tokenizer_only=True, checkpoint_dir=tmp_path)

     torch.manual_seed(123)
     llm = LLM.load(
         model="pythia-14m",
         init="random",
         tokenizer_dir=Path(tmp_path/"EleutherAI/pythia-14m")
     )

Works fine locally though ...

[rasbt]

To narrow it down further, it only happens with the default settings, not when distribute=None. This means, it's something related to the code here:

litgpt/litgpt/api.py

Lines 215 to 239 in a686b40

	if distribute == "auto":
	if torch.cuda.is_available():
	accelerator = "cuda"
	elif torch.backends.mps.is_available():
	accelerator = "mps"
	else:
	accelerator = "cpu"
	fabric = L.Fabric(
	accelerator=accelerator,
	devices=1,
	precision=get_default_supported_precision(training=False),
	)
	with fabric.init_module(empty_init=False):
	model = GPT(config)
	model.eval()
	preprocessor = Preprocessor(tokenizer, device=fabric.device)
	if checkpoint_dir is not None:
	checkpoint_path = checkpoint_dir / "lit_model.pth"
	check_file_size_on_cpu_and_warn(checkpoint_path, fabric.device)
	load_checkpoint(fabric, model, checkpoint_path)
	model = fabric.setup_module(model)

Perhaps the MPS support in the CI has some issues. (Since it works fine locally.)

[rasbt]

@t-vi @Andrei-Aksionov

Ah, so it does seem to be MPS related. I.e., changing

     if torch.cuda.is_available(): 
         accelerator = "cuda" 
     elif torch.backends.mps.is_available(): 
         accelerator = "mps" 
     else: 
         accelerator = "cpu" 

to

     if torch.cuda.is_available(): 
         accelerator = "cuda" 
     else: 
         accelerator = "cpu" 

will fix those tests on the the CI. My guess is that's it's something particular about the CI machine because it works fine locally on 2 of my Macs (+ also on Andrei's Mac). Maybe outdated drivers.

So let's just skip MPS-related tests on that machine.

[rasbt]

Ok, I left MPS disabled for the macos runner since it seems to have issues. Could be Fabric-related, driver-related or LitGPT-related (although it works fine locally). Let's merge this for now and revisit in a few weeks or months when the macos-15 machines are more readily available in workflows. Maybe their drivers are just old.