---
title: NLTokenizer
framework: naturallanguage
role: symbol
role_heading: Class
path: naturallanguage/nltokenizer
---

# NLTokenizer

A tokenizer that segments natural language text into semantic units.

## Declaration

```swift
class NLTokenizer
```

## Mentioned in

Tokenizing natural language text

## Overview

Overview NLTokenizer creates individual units from natural language text. Define the desired unit (word, sentence, paragraph, or document as declared in the NLTokenUnit) for tokenization, and then assign a string to tokenize. The enumerateTokensInRange:usingBlock: method provides the ranges of the tokens in the string based on the tokenization unit. For more information, see Tokenizing natural language text. important: Use an NLTokenizer instance on one thread or one dispatch queue at a time. You do this by either serializing method calls to the tokenizer, or by creating a separate tokenizer instance for each thread and dispatch queue.

## Topics

### Creating a tokenizer

- [init(unit:)](naturallanguage/nltokenizer/init(unit:).md)

### Configuring a tokenizer

- [string](naturallanguage/nltokenizer/string.md)
- [setLanguage(_:)](naturallanguage/nltokenizer/setlanguage(_:).md)
- [unit](naturallanguage/nltokenizer/unit.md)
- [NLTokenizer.Attributes](naturallanguage/nltokenizer/attributes.md)

### Enumerating the tokens

- [enumerateTokens(in:using:)](naturallanguage/nltokenizer/enumeratetokens(in:using:).md)
- [tokens(for:)](naturallanguage/nltokenizer/tokens(for:).md)
- [tokenRange(at:)](naturallanguage/nltokenizer/tokenrange(at:).md)
- [tokenRange(for:)](naturallanguage/nltokenizer/tokenrange(for:).md)

## Relationships

### Inherits From

- [NSObject](objectivec/nsobject-swift.class.md)

### Conforms To

- [CVarArg](swift/cvararg.md)
- [CustomDebugStringConvertible](swift/customdebugstringconvertible.md)
- [CustomStringConvertible](swift/customstringconvertible.md)
- [Equatable](swift/equatable.md)
- [Hashable](swift/hashable.md)
- [NSObjectProtocol](objectivec/nsobjectprotocol.md)

## See Also

### Tokenization

- [Tokenizing natural language text](naturallanguage/tokenizing-natural-language-text.md)