mlx-swift-examples prep for mlx-swift-lm 3.x release#468
Conversation
| from: #hubDownloader(), | ||
| using: #huggingFaceTokenizerLoader(), | ||
| configuration: modelConfiguration | ||
| ) { value in |
There was a problem hiding this comment.
Along with linking the HuggingFace libraries, this is the pattern for adopting the new API.
| let downloader = #hubDownloader() | ||
|
|
||
| let resolved = try await resolve( | ||
| configuration: modelConfiguration, from: downloader, useLatest: false |
There was a problem hiding this comment.
This is a little more complex because it separates out the download from the load. resolve() will download and produce resolved urls that can be loaded.
| let downloader = #hubDownloader() | ||
| let loader = #huggingFaceTokenizerLoader() |
There was a problem hiding this comment.
This doesn't work yet -- there is a local Progress type that conflicts with the use in the macro. See ml-explore/mlx-swift-lm#189
| isEnabled = "NO"> | ||
| </CommandLineArgument> | ||
| <CommandLineArgument | ||
| argument = " --download ~/Downloads/huggingface" | ||
| isEnabled = "YES"> | ||
| </CommandLineArgument> | ||
| <CommandLineArgument | ||
| argument = "--model mlx-community/GLM-4.7-Flash-4bit --max-tokens 5000 --prompt "Testing, testing… Who are you? Identify yourself"" |
There was a problem hiding this comment.
- remove, from local testing
| <EnvironmentVariables> | ||
| <EnvironmentVariable | ||
| key = "METAL_DEVICE_WRAPPER_TYPE" | ||
| value = "1" | ||
| isEnabled = "YES"> | ||
| </EnvironmentVariable> | ||
| </EnvironmentVariables> |
| @Flag(name: .customLong("tool-time"), help: "Enable time telling tool") | ||
| var useTimeTool = false |
There was a problem hiding this comment.
A way to try the tool integration -- this has better support now.
| // TODO maybe just use ChatSession here? | ||
| let x = try await call(toolCall: toolCall) |
There was a problem hiding this comment.
- replace with ChatSession? it has the tool call loop already implemented
| let cache = | ||
| if let download = args.args.download { | ||
| HubCache(cacheDirectory: download) | ||
| } else { | ||
| HubCache.default | ||
| } | ||
|
|
||
| let parts = output.components(separatedBy: "/") | ||
| guard parts.count == 2 else { | ||
| fatalError("output must be org/name, e.g. mlx-community/mistral-lora: \(output)") | ||
| } | ||
| let repo = Repo.ID(namespace: parts[0], name: parts[1]) | ||
| outputURL = cache.repoDirectory(repo: repo, kind: .model) |
There was a problem hiding this comment.
Maybe tis should be a call to resolve()?
davidkoski
force-pushed
the
mlx-swift-lm-3
branch
from
72dae6c to
b146473
Compare
davidkoski
force-pushed
the
mlx-swift-lm-3
branch
from
c8bd8bb to
e091e33
Compare
angeloskath
left a comment
angeloskath
left a comment
There was a problem hiding this comment.
Looks good.
There is a random edit in the tutorial so I left a comment for it but otherwise 👍
|
|
||
| // make an array of shape [2, 2] filled with ones | ||
| let y = MLXArray.ones([2, 2]) | ||
| let y = MLXArray.ones([2, 2], type: Float.self) |
There was a problem hiding this comment.
Shouldn't that be the default?
There was a problem hiding this comment.
Yes, actually it is -- I think I picked this up when building against this:
There is a thread of investigation there, but in the end I am not convinced it is really what is going on.
2 participants
Proposed changes
This is preparation for the mlx-swift-lm 3.x release (new APIs). This is ready to go. I need to pick up the tag when it is ready.
FYI @DePasqualeOrg
Checklist
Put an
xin the boxes that apply.pre-commit run --all-filesto format my code / installed pre-commit prior to committing changes