---
title: CFStringTokenizerTokenType
framework: corefoundation
role: symbol
role_heading: Structure
path: corefoundation/cfstringtokenizertokentype
---

# CFStringTokenizerTokenType

Token types returned by CFStringTokenizerGoToTokenAtIndex(_:_:) and CFStringTokenizerAdvanceToNextToken(_:).

## Declaration

```swift
struct CFStringTokenizerTokenType
```

## Overview

Overview See http://www.unicode.org/reports/tr29/#Word_Boundaries for a detailed description of word boundaries.

## Topics

### Constants

- [normal](corefoundation/cfstringtokenizertokentype/normal.md)
- [hasSubTokensMask](corefoundation/cfstringtokenizertokentype/hassubtokensmask.md)
- [hasDerivedSubTokensMask](corefoundation/cfstringtokenizertokentype/hasderivedsubtokensmask.md)
- [hasHasNumbersMask](corefoundation/cfstringtokenizertokentype/hashasnumbersmask.md)
- [hasNonLettersMask](corefoundation/cfstringtokenizertokentype/hasnonlettersmask.md)
- [isCJWordMask](corefoundation/cfstringtokenizertokentype/iscjwordmask.md)

### Initializers

- [init(rawValue:)](corefoundation/cfstringtokenizertokentype/init(rawvalue:).md)

## Relationships

### Conforms To

- [BitwiseCopyable](swift/bitwisecopyable.md)
- [Equatable](swift/equatable.md)
- [ExpressibleByArrayLiteral](swift/expressiblebyarrayliteral.md)
- [OptionSet](swift/optionset.md)
- [RawRepresentable](swift/rawrepresentable.md)
- [Sendable](swift/sendable.md)
- [SendableMetatype](swift/sendablemetatype.md)
- [SetAlgebra](swift/setalgebra.md)

## See Also

### Constants

- [Tokenization Modifiers](corefoundation/1588024-tokenization-modifiers.md)