spencer0124/sentencepiecekit
A Swift package wrapper for Google's [SentencePiece](https://github.com/google/sentencepiece), allowing you to use SentencePiece tokenizer directly in your **iOS**, **macOS**, and **watchOS** projects.
π Project Structure
To use the build script, your directory structure must look like this:
Projects/ # Verification Root
βββ sentencepiece/ # Cloned Google SentencePiece repository
βββ SentencePieceKit/ # This project
βββ Scripts/
β βββ build_xcframework.sh
βββ Package.swiftNote: The script assumes
sentencepieceandSentencePieceKitare sibling directories.
π Prerequisites
Ensure you have the following tools installed on your Mac:
- Xcode (with Command Line Tools)
- CMake (
brew install cmake) - wget (
brew install wget) - Used to download the iOS toolchain
π How to Build
- Clone Repositories:
```bash # Go to your working directory cd Projects
# 1. Clone Google's SentencePiece git clone https://github.com/google/sentencepiece.git
# 2. Clone SentencePieceKit (if you haven't already) git clone https://github.com/your-repo/SentencePieceKit.git ```
- Run the Build Script:
``bash cd SentencePieceKit/Scripts chmod +x build_xcframework.sh ./build_xcframework.sh ``
- Wait for Completion:
The script will: - Clean up previous builds. - Automatically download the necessary ios.toolchain.cmake. - Patch CMakeLists.txt for watchOS compatibility. - Compile for all platforms and architectures. - Generate sentencepiece.xcframework.
- Verify:
After the build finishes, you will see sentencepiece.xcframework in the root of SentencePieceKit. You can run the verification script to ensure all architectures are present: ``bash ./verify_kit.sh ``
π¦ Installation in Xcode
- Open your project in Xcode.
- Go to File > Add Package Dependencies...
- Add local
SentencePieceKitpackage or dragsentencepiece.xcframeworkmanually if you prefer direct linking. - Add
import SentencePieceKitin your Swift code.
π License
- SentencePiece: Apache License 2.0 (Google)
- SentencePieceKit: Apache License 2.0
Package Metadata
Repository: spencer0124/sentencepiecekit
Default branch: main
README: README.md