strtok, strtok_s
From cppreference.com
| Defined in header <string.h>
|
||
| (1) | ||
char* strtok( char* str, const char* delim );
|
(until C99) | |
char* strtok( char* restrict str, const char* restrict delim );
|
(since C99) | |
char* strtok_s( char* restrict str, rsize_t* restrict strmax,
const char* restrict delim, char** restrict ptr );
|
(2) | (since C11) |
Tokenizes a null-terminated byte string.
1) A sequence of calls to
strtok breaks the string pointed to by str into a sequence of tokens, each of which is delimited by a character from the string pointed to by delim. Each call in the sequence has a search target :
- If
stris non-null, the call is the first call in the sequence. The search target is null-terminated byte string pointed to bystr. - If
stris null, the call is one of the subsequent calls in the sequence. The search target is determined by the previous call in the sequence.
Each call in the sequence searches the search target for the first character that is not contained in the separator string pointed to by
delim, the separator string can be different from call to call.
- If no such character is found, then there are no tokens in the search target. The search target for the next call in the sequence is unchanged.[1]
- If such a character is found, it is the start of the current token.
strtokthen searches from there for the first character that is contained in the separator string.- If no such character is found, the current token extends to the end of search target. The search target for the next call in the sequence is an empty string.[2]
- If such a character is found, it is overwritten by a null character, which terminates the current token. The search target for the next call in the sequence starts from the following character.
If
str or delim is not a pointer to a null-terminated byte string, the behavior is undefined.2) Same as (1), except for the following differences:
- In each call, writes the number of characters left to see in
strinto*strmaxand writes the tokenizer's internal state into*ptr. - Subsequent calls in the sequence must pass
strmaxandptrwith the values stored by the previous call. - The following errors are detected at runtime and call the currently installed constraint handler function, without storing anything in the object pointed to by
ptr:strmax,delim, orptris a null pointer.*ptris a null pointer for a subsequent call in the sequence.*strmaxis greater thanRSIZE_MAX.- The end of the token found does not occur within the first
*s1maxcharacters of the search target.
If both
str points to a character array which lacks the null character and strmax points to a value which is greater than the size of that character array, the behavior is undefined. As with all bounds-checked functions,
strtok_s is only guaranteed to be available if __STDC_LIB_EXT1__ is defined by the implementation and if the user defines __STDC_WANT_LIB_EXT1__ to the integer constant 1 before including <string.h>.Parameters
| str | - | pointer to the null-terminated byte string to tokenize |
| delim | - | pointer to the null-terminated byte string identifying delimiters |
| strmax | - | pointer to an object which initially holds the size of str: strtok_s stores the number of characters that remain to be examined
|
| ptr | - | pointer to an object of type char*, which is used by strtok_s to store its internal state
|
Return value
1) Returns a pointer to the first character of the next token, or a null pointer if there is no token.
2) Returns a pointer to the first character of the next token, or a null pointer if there is
no token or there is a runtime-constraint violation.
Note
This function is destructive: it writes the '\0' characters in the elements of the string str. In particular, a string literal cannot be used as the first argument of strtok.
Each call to strtok modifies a static variable: is not thread safe.
Unlike most other tokenizers, the delimiters in strtok can be different for each subsequent token, and can even depend on the contents of the previous tokens.
The strtok_s function differs from the POSIX strtok_r function by guarding against storing outside of the string being tokenized, and by checking runtime constraints. The Microsoft CRT strtok_s signature matches this POSIX strtok_r definition, not the C11 strtok_s.
Example
Run this code
#define __STDC_WANT_LIB_EXT1__ 1
#include <stdio.h>
#include <string.h>
int main(void)
{
char input[] = "A bird came down the walk";
printf("Parsing the input string '%s'\n", input);
char* token = strtok(input, " ");
while (token)
{
puts(token);
token = strtok(NULL, " ");
}
printf("Contents of the input string now: '");
for (size_t n = 0; n < sizeof input; ++n)
input[n] ? putchar(input[n]) : fputs("\\0", stdout);
puts("'");
#ifdef __STDC_LIB_EXT1__
char str[] = "A bird came down the walk";
rsize_t strmax = sizeof str;
const char* delim = " ";
char* next_token;
printf("Parsing the input string '%s'\n", str);
token = strtok_s(str, &strmax, delim, &next_token);
while (token)
{
puts(token);
token = strtok_s(NULL, &strmax, delim, &next_token);
}
printf("Contents of the input string now: '");
for (size_t n = 0; n < sizeof str; ++n)
str[n] ? putchar(str[n]) : fputs("\\0", stdout);
puts("'");
#endif
}
Possible output:
Parsing the input string 'A bird came down the walk'
A
bird
came
down
the
walk
Contents of the input string now: 'A\0bird\0came\0down\0the\0walk\0'
Parsing the input string 'A bird came down the walk'
A
bird
came
down
the
walk
Contents of the input string now: 'A\0bird\0came\0down\0the\0walk\0'
References
- C23 standard (ISO/IEC 9899:2024):
- 7.24.5.8 The strtok function (p: TBD)
- K.3.7.3.1 The strtok_s function (p: TBD)
- C17 standard (ISO/IEC 9899:2018):
- 7.24.5.8 The strtok function (p: TBD)
- K.3.7.3.1 The strtok_s function (p: TBD)
- C11 standard (ISO/IEC 9899:2011):
- 7.24.5.8 The strtok function (p: 369-370)
- K.3.7.3.1 The strtok_s function (p: 620-621)
- C99 standard (ISO/IEC 9899:1999):
- 7.21.5.8 The strtok function (p: 332-333)
- C89/C90 standard (ISO/IEC 9899:1990):
- 4.11.5.8 The strtok function