---

ACE_Tokenizer Class Reference

Tokenizer. More...

#include <SString.h>

List of all members.

Public Types
enum	{ MAX_DELIMITERS = 16, MAX_PRESERVES = 16 }
Public Methods
	ACE_Tokenizer (ACE_TCHAR *buffer)
int	delimiter (ACE_TCHAR d)
int	delimiter_replace (ACE_TCHAR d, ACE_TCHAR replacement)
int	preserve_designators (ACE_TCHAR start, ACE_TCHAR stop, int strip=1)
ACE_TCHAR *	next (void)
	Returns the next token. More...
Protected Methods
int	is_delimiter (ACE_TCHAR d, int &replace, ACE_TCHAR &r)
	Returns 1 if <d> is a delimiter, 0 otherwise. If <d> should be replaced with <r>, <replace> is set to 1, otherwise 0. More...
int	is_preserve_designator (ACE_TCHAR start, ACE_TCHAR &stop, int &strip)
Protected Attributes
ACE_TCHAR *	buffer_
int	index_
Preserve_Entry	preserves_ [MAX_PRESERVES]
	The application can specify MAX_PRESERVES preserve designators. More...
int	preserves_index_
	Pointer to the next free spot in preserves_. More...
Delimiter_Entry	delimiters_ [MAX_DELIMITERS]
	The tokenizer allows MAX_DELIMITERS number of delimiters. More...
int	delimiter_index_
	Pointer to the next free space in delimiters_. More...

---

Detailed Description

Tokenizer.

Tokenizes a buffer. Allows application to set delimiters and preserve designators. Does not allow special characters, yet (e.g., printf ("\"like a quoted string\"")).

Definition at line 258 of file SString.h.

---

Member Enumeration Documentation

anonymous enum

Enumeration values: MAX_DELIMITERS  MAX_PRESERVES  Definition at line 368 of file SString.h. { MAX_DELIMITERS=16, MAX_PRESERVES=16 };

---

Constructor & Destructor Documentation

ACE_Tokenizer::ACE_Tokenizer (  ACE_TCHAR *    buffer )

buffer will be parsed. Notice that ACE_Tokenizer will modify buffer if you use delimiter_replace or preserve_designators to do character substitution. NOTE: You should NOT pass a constant string or string literal to this constructor, since ACE_Tokenizer will try to modify the string. See also: preserve_designators , preserve_designators Definition at line 21 of file SString.cpp. References ACE_TCHAR. : buffer_ (buffer), index_ (0), preserves_index_ (0), delimiter_index_ (0) { }

---

Member Function Documentation

int ACE_Tokenizer::delimiter (  ACE_TCHAR    d )

d is a delimiter. Returns: Returns 0 on success, -1 if there is no memory left. Example: char buf[30]; ACE_OS::strcpy(buf, "William/Joseph/Hagins"); ACE_Tokenizer tok (buf); tok.delimiter ('/'); for (char *p = tok.next (); p; p = tok.next ()) cout << p << endl; This will print out: William/Joseph/Hagins Joseph/Hagins Hagins Definition at line 30 of file SString.cpp. References ACE_TCHAR, delimiter_index_, delimiters_, and MAX_DELIMITERS. Referenced by ACE_Sock_Connect::get_ip_interfaces. { if (delimiter_index_ == MAX_DELIMITERS) return -1; delimiters_[delimiter_index_].delimiter_ = d; delimiters_[delimiter_index_].replace_ = 0; delimiter_index_++; return 0; }

int ACE_Tokenizer::delimiter_replace (  ACE_TCHAR    d, ACE_TCHAR    replacement )

d is a delimiter and, when found, will be replaced by replacement. Returns: 0 on success, -1 if there is no memory left. Example: char buf[30]; ACE_OS::strcpy(buf, "William/Joseph/Hagins"); ACE_Tokenizer tok (buf); tok.delimiter_replace ('/', 0); for (char *p = tok.next (); p; p = tok.next ()) cout << p << endl; This will print out: William Joseph Hagins Definition at line 42 of file SString.cpp. References ACE_TCHAR, ACE_Tokenizer::Delimiter_Entry::delimiter_, delimiter_index_, delimiters_, MAX_DELIMITERS, ACE_Tokenizer::Delimiter_Entry::replace_, and ACE_Tokenizer::Delimiter_Entry::replacement_. Referenced by ACE_Process_Options::command_line_argv, and ACE_Configuration::expand_path. { // Make it possible to replace delimiters on-the-fly, e.g., parse // string until certain token count and then copy rest of the // original string. for (int i = 0; i < delimiter_index_; i++) if (delimiters_[i].delimiter_ == d) { delimiters_[i].replacement_ = replacement; delimiters_[i].replace_ = 1; return 0; } if (delimiter_index_ >= MAX_DELIMITERS) return -1; delimiters_[delimiter_index_].delimiter_ = d; delimiters_[delimiter_index_].replacement_ = replacement; delimiters_[delimiter_index_].replace_ = 1; delimiter_index_++; return 0; }

int ACE_Tokenizer::is_delimiter (  ACE_TCHAR    d, int &    replace, ACE_TCHAR &    r )  [protected]

Returns 1 if <d> is a delimiter, 0 otherwise. If <d> should be replaced with <r>, <replace> is set to 1, otherwise 0. Definition at line 82 of file SString.cpp. References ACE_TCHAR, ACE_Tokenizer::Delimiter_Entry::delimiter_, delimiter_index_, delimiters_, ACE_Tokenizer::Delimiter_Entry::replace_, and ACE_Tokenizer::Delimiter_Entry::replacement_. Referenced by next. { replace = 0; for (int x = 0; x < delimiter_index_; x++) if (delimiters_[x].delimiter_ == d) { if (delimiters_[x].replace_) { r = delimiters_[x].replacement_; replace = 1; } return 1; } return 0; }

int ACE_Tokenizer::is_preserve_designator (  ACE_TCHAR    start, ACE_TCHAR &    stop, int &    strip )  [protected]

If <start> is a start preserve designator, returns 1 and sets <stop> to the stop designator. Returns 0 if <start> is not a preserve designator. Definition at line 103 of file SString.cpp. References ACE_TCHAR, preserves_, preserves_index_, ACE_Tokenizer::Preserve_Entry::start_, ACE_Tokenizer::Preserve_Entry::stop_, and ACE_Tokenizer::Preserve_Entry::strip_. Referenced by next. { for (int x = 0; x < preserves_index_; x++) if (preserves_[x].start_ == start) { stop = preserves_[x].stop_; strip = preserves_[x].strip_; return 1; } return 0; }

ACE_TCHAR * ACE_Tokenizer::next (  void    )

Returns the next token. Definition at line 119 of file SString.cpp. References ACE_TCHAR, buffer_, index_, is_delimiter, and is_preserve_designator. Referenced by ACE_Process_Options::command_line_argv, ACE_Configuration::expand_path, and ACE_Sock_Connect::get_ip_interfaces. { // Check if the previous pass was the last one in the buffer. if (index_ == -1) { index_ = 0; return 0; } ACE_TCHAR replacement; int replace; ACE_TCHAR *next_token; // Skip all leading delimiters. for (;;) { // Check for end of string. if (buffer_[index_] == '\0') { // If we hit EOS at the start, return 0. index_ = 0; return 0; } if (this->is_delimiter (buffer_[index_], replace, replacement)) index_++; else break; } // When we reach this point, buffer_[index_] is a non-delimiter and // not EOS - the start of our next_token. next_token = buffer_ + index_; // A preserved region is it's own token. ACE_TCHAR stop; int strip; if (this->is_preserve_designator (buffer_[index_], stop, strip)) { while (++index_) { if (buffer_[index_] == '\0') { index_ = -1; goto EXIT_LABEL; } if (buffer_[index_] == stop) break; } if (strip) { // Skip start preserve designator. next_token += 1; // Zap the stop preserve designator. buffer_[index_] = '\0'; // Increment to the next token. index_++; } goto EXIT_LABEL; } // Step through finding the next delimiter or EOS. for (;;) { // Advance pointer. index_++; // Check for delimiter. if (this->is_delimiter (buffer_[index_], replace, replacement)) { // Replace the delimiter. if (replace != 0) buffer_[index_] = replacement; // Move the pointer up and return. index_++; goto EXIT_LABEL; } // A preserve designator signifies the end of this token. if (this->is_preserve_designator (buffer_[index_], stop, strip)) goto EXIT_LABEL; // Check for end of string. if (buffer_[index_] == '\0') { index_ = -1; goto EXIT_LABEL; } } EXIT_LABEL: return next_token; }

int ACE_Tokenizer::preserve_designators (  ACE_TCHAR    start, ACE_TCHAR    stop, int    strip = 1 )

Extract string between a pair of designator characters. For instance, quotes, or '(' and ')'. start specifies the begin designator. stop specifies the end designator. strip If strip == 1, then the preserve designators will be stripped from the tokens returned by next. Returns: 0 on success, -1 if there is no memory left. Example with strip = 0: char buf[30]; ACE_OS::strcpy(buf, "William(Joseph)Hagins"); ACE_Tokenizer tok (buf); tok.preserve_designators ('(', ')', 0); for (char *p = tok.next (); p; p = tok.next ()) cout << p << endl; This will print out: William(Joseph)Hagins (Joseph)Hagins )Hagins Example with strip = 1: char buf[30]; ACE_OS::strcpy(buf, "William(Joseph)Hagins"); ACE_Tokenizer tok (buf); tok.preserve_designators ('(', ')', 1); for (char *p = tok.next (); p; p = tok.next ()) cout << p << endl; This will print out: William Joseph Hagins Definition at line 67 of file SString.cpp. References ACE_TCHAR, MAX_PRESERVES, preserves_, and preserves_index_. Referenced by ACE_Process_Options::command_line_argv. { if (preserves_index_ == MAX_PRESERVES) return -1; preserves_[preserves_index_].start_ = start; preserves_[preserves_index_].stop_ = stop; preserves_[preserves_index_].strip_ = strip; preserves_index_++; return 0; }

---

Member Data Documentation

ACE_TCHAR* ACE_Tokenizer::buffer_ [protected]

Definition at line 385 of file SString.h. Referenced by next.

int ACE_Tokenizer::delimiter_index_ [protected]

Pointer to the next free space in delimiters_. Definition at line 445 of file SString.h. Referenced by delimiter, delimiter_replace, and is_delimiter.

Delimiter_Entry ACE_Tokenizer::delimiters_[MAX_DELIMITERS] [protected]

The tokenizer allows MAX_DELIMITERS number of delimiters. Definition at line 442 of file SString.h. Referenced by delimiter, delimiter_replace, and is_delimiter.

int ACE_Tokenizer::index_ [protected]

Definition at line 386 of file SString.h. Referenced by next.

Preserve_Entry ACE_Tokenizer::preserves_[MAX_PRESERVES] [protected]

The application can specify MAX_PRESERVES preserve designators. Definition at line 414 of file SString.h. Referenced by is_preserve_designator, and preserve_designators.

int ACE_Tokenizer::preserves_index_ [protected]

Pointer to the next free spot in preserves_. Definition at line 417 of file SString.h. Referenced by is_preserve_designator, and preserve_designators.

---

The documentation for this class was generated from the following files:

• SString.h
• SString.cpp

---