NLTokenUnit.paragraph

Declaration

case paragraph

Discussion

Use this linguistic unit to tokenize text into paragraphs, like in the following example:

let text = "This is a sentence. This is another sentence.\nThis is a second paragraph."

let tokenizer = NLTokenizer(unit: .paragraph)
tokenizer.string = text

let range = text.startIndex..<text.endIndex

let tokenArray = tokenizer.tokens(for: range)
print("Number of tokens: \(tokenArray.count)")

tokenizer.enumerateTokens(in: range) { tokenRange, _ in
    print(text[tokenRange])
    return true
}

For more information, see Tokenizing natural language text.

See Also

Constants

• NLTokenUnit.word
• NLTokenUnit.sentence
• NLTokenUnit.document