Contents

init(unit:)

Creates a tokenizer with the specified unit.

Declaration

init(unit: NLTokenUnit)

Parameters

  • unit:

    The level of granularity that the tokenizer operates at.