1 # <a name="Search Pattern Cookbook"></a><a name=" Search Pattern Cookbook"></a> Search Pattern Cookbook
3 The Search function in TWiki is very powerful. Especially searches using a [[RegularExpression]] play an important part of tapping TWiki's full potential. Unfortunately [[RegularExpressions]] can be incredibly obscure to the uninitiated.
5 Most people not familiar (enough) with Regular Expressions mostly cut and paste (and maybe tweak) from existing examples. This page intends to collect lots of examples together.
11 <li><a href="#Pattern 1: Extract values from a"> Pattern 1: Extract values from a table</a></li>
14 <li><a href="#TWiki Installation Error">TWiki Installation Error</a><ul>
15 <li><a href="#Pattern 2: List generated from f"> Pattern 2: List generated from form classification</a></li>
16 <li><a href="#Pattern 3: Creating lists of TWi"> Pattern 3: Creating lists of TWiki usernames</a></li>
19 <li><a href="#TWiki Installation Error">TWiki Installation Error</a></li>
20 <li><a href="#TWiki Installation Error">TWiki Installation Error</a><ul>
21 <li><a href="#Pattern 4: Extract the parent of"> Pattern 4: Extract the parent of a given topic</a></li>
24 <li><a href="#TWiki Installation Error">TWiki Installation Error</a></li>
25 <li><a href="#TWiki Installation Error">TWiki Installation Error</a><ul>
26 <li><a href="#Pattern 5: Show all Children of"> Pattern 5: Show all Children of a given topic</a></li>
27 <li><a href="#Pattern 6: Search and display th"> Pattern 6: Search and display the home topics of public webs in a list</a></li>
30 <li><a href="#TWiki Installation Error">TWiki Installation Error</a><ul>
31 <li><a href="#Pattern 7: Create a select box w"> Pattern 7: Create a select box with values from a bullet list</a></li>
32 <li><a href="#Pattern 8: Extract a value from"> Pattern 8: Extract a value from a named bullet list item</a></li>
35 <li><a href="#TWiki Installation Error">TWiki Installation Error</a><ul>
36 <li><a href="#Pattern 9: Search for Form and M"> Pattern 9: Search for Form and Meta data: explained</a></li>
37 <li><a href="#Pattern 10: Search all topics th"> Pattern 10: Search all topics that have been moved</a></li>
43 <a name="SearchTables"></a>
45 ## <a name="Pattern 1: Extract values from a"></a> Pattern 1: Extract values from a table
47 ### <a name="Problem definition"></a> Problem definition
49 Suppose there is a topic with a table defining entries in a TWikiForm. I.e. they define select menu items in a form template. They are then formatted like:
51 | *Name* | *Type* | *Tooltip message* |
52 | option1 | option | |
53 | option2 | option | |
54 | option3 | option | |
56 How to extract the 'name' values, i.e. 'option1', 'option2' and 'option3' and put them in a HTML form select input?
58 ### <a name="Solution"></a> Solution
60 The following search pattern can be employed:
64 %SEARCH{ "^\|[^\|]*\| *option *\|" topic="%TOPIC%" type="regex" multiple="on" nosearch="on" nototal="on" format="<option>$pattern(^\| *(.*?) *\|.*)</option>" }%
70 <form><select><h1><a name="TWiki Installation Error"></a> TWiki Installation Error </h1>Incorrect format of searchformat template (missing sections? There should be 4 %SPLIT% tags) </select></form>
72 <a name="SearchFormClassification"></a>
74 ## <a name="Pattern 2: List generated from f"></a> Pattern 2: List generated from form classification
76 ### <a name="Problem"></a> Problem
78 Imagine a TWiki form-based topic classification, i.e. every page has a form with several fields. How to:
80 1. create a search to display all topics where one form field is set to a certain value
81 2. create a search to filter the list above based on the values of a second form field
83 ### <a name="Test case"></a> Test case
85 In practice: %BR% Image a TWiki form with two fields:
87 - TopicClassification = One, Two or Three
88 - TopicStatus = Test or Final
92 1. List all topics where the TopicClassification field is set to 'Two'
93 2. Enable the user to filter this list based on the values of TopicStatus
95 ### <a name="Solution"></a> Solution
97 %SEARCH{"[T]opicClassification.*value\=.*Two;[T]opicStatus.*value\=.*%URLPARAM{type}%"
98 type="regex" casesensitive="on" nosearch="on"
99 format=" * $topic - <font face=\"arial,helvetica\" size=\"1\">
100 _last modified by_ $wikiusername _on_ $date </font> %BR%
101 <font face=\"arial,helvetica\" size=\"1\"> $formfield(TopicStatus) </font>"
104 The filtering select dialogue is created as in Pattern 1:
108 <form name="selectType" action="%SCRIPTURLPATH{"view"}%/%WEB%/" >
109 <select name="type" size="1" onchange="document.location=this.value;">
110 %SEARCH{ "^\|[^\|]*\| *option *\|" topic="TopicClassification" web="%WEB%" type="regex"
111 multiple="on" nosearch="on" nototal="on" format="<option value=%INCLUDINGTOPIC%?type=$pattern(^\| *(.*?) *\|.*)>$pattern(^\| *(.*?) *\|.*)</option>" }%
112 <option value=%INCLUDINGTOPIC%>All pages</option> </select>
116 This will create similar functionality as TWiki:Plugins.TopicClassificationAddOn
118 <a name="SearchUsernames"></a>
120 ## <a name="Pattern 3: Creating lists of TWi"></a> Pattern 3: Creating lists of TWiki usernames
122 ### <a name="Problem"></a> Problem
124 How to populate a list box with all usernames of registered TWiki users
126 ### <a name="Solution 1: Appropriate for Sep"></a><a name="Solution 1: Appropriate for Sep "></a> Solution 1: Appropriate for Sep 2004 TWiki (Cairo)
128 <form name="testing" action="%SCRIPTURLPATH{"view"}%/%USERSWEB%" method="get">
129 <select name="topic">
130 <option>Select user...</option>
131 %SEARCH{ "Name:;Email:;Country:" web="%USERSWEB%" type="regex" nosearch="on" nototal="on" format="<option>$topic</option>" }%
133 <input type="submit" value="Go" />
136 Which expands to this: (here limited to all Z\* users because TWiki.org has so many)
138 <form action="http://www.dementia.org/twiki/view/%USERSWEB%" method="get" name="testing"><select name="topic"><option>Select user...</option>
139 <h1><a name="TWiki Installation Error"></a> TWiki Installation Error </h1>Incorrect format of searchformat template (missing sections? There should be 4 %SPLIT% tags) </select> <input type="submit" value="Go" /></form>
141 This searches all topics in the Main web that contain "Name", "Email" and "Country" bullets. Alternatively, do a %SYSTEMWEB%.FormattedSearch with `multiple="on"` on the [[%USERSWEB%.TWikiUsers|USERSWEB/TWikiUsers]] topic.
143 ### <a name="Solution 2: As Solution 1, but w"></a> Solution 2: As Solution 1, but with possibility for multi-selecting usernames
145 The example of Solution 1 produces the list box. Add a MULTIPLE to the _select_ statement, i.e.:
147 <select name="topic" size="2" MULTIPLE>
149 Please note that the Search pattern is unchanged compared to Solution 1. The change is in the HTML form element.
151 The abovementioned modification is, in effect:
153 <form action="http://www.dementia.org/twiki/view/%USERSWEB%" method="get" name="testing"><select multiple="MULTIPLE" name="topic" size="2"><option>Select user...</option>
154 <h1><a name="TWiki Installation Error"></a> TWiki Installation Error </h1>Incorrect format of searchformat template (missing sections? There should be 4 %SPLIT% tags) </select> <input type="submit" value="Go" /></form>
156 ### <a name="Solution 3: Appropriate for TWik"></a> Solution 3: Appropriate for TWiki 4 (Dakar)
158 When the User information is stored in a [[UserForm]] (as is default in Dakar) then this list can be generated as follows:
160 <form name="testing" action="%SCRIPTURLPATH{"view"}%/%USERSWEB%" method="get">
161 <select name="topic">
162 <option>Select user...</option>
163 %SEARCH{"%META:FORM.*[U]serForm" web="%USERSWEB%" type="regex" casesensitive="on" nosearch="on" format="<option>$topic</option>" sort="topic" excludetopic="Test*, TWiki*"}%
165 <input type="submit" value="Go" />
168 In the above example:
170 - <code>**META:FORM.\*[U]serForm**</code> will search for all topics with a [[UserForm]] attached - change this if you have a different form where userdata is stored. Please note that this search does not actually extract anything from the form - it just uses it to identify the appropriate pages
171 - <code>**excludetopic="Test\*, TWiki\*"**</code> allows to skip all topics starting with Test and TWiki, such as TestUser or TWikiAdmin. Use this if you have any special users who you do not want appearing in this list
173 <a name="SearchTopicParent"></a>
175 ## <a name="Pattern 4: Extract the parent of"></a> Pattern 4: Extract the parent of a given topic
177 ### <a name="Problem"></a> Problem
179 How to get to the parent of the current topic to display on the page?
181 ### <a name="Solution 1: Using META"></a> Solution 1: Using META
183 Since TWiki 4.0 you can now use the META variable:
185 `%META{ "parent" dontrecurse="on" }%`
187 ### <a name="Solution 2: Using _SpreadSheetPl"></a> Solution 2: Using SpreadSheetPlugin
189 You might think that the following Search would do the trick:
191 %SEARCH{ "^%BASETOPIC%$" scope="topic" nosearch="on" type="regex" nonoise="on" format=" * $parent" }%
193 However, the `$parent` link fails if the topic has no parent set (`$parent` will be empty). You can use some TWiki:Plugins/SpreadSheetPlugin magic to conditionally link to the parent or to `WebHome`:
195 $percntCALC{$IF($EXACT($parent,),<nop>,$NOP( * $parent))}$percnt
197 So the total Search query to find a topic's parent topic is:
199 %SEARCH{ "^%BASETOPIC%$" scope="topic" type="regex" nonoise="on" format="$percntCALC{$IF($EXACT($parent,),<nop>,$NOP( * $parent))}$percnt" }%
201 #### <a name="Test Case"></a> Test Case
203 The parent topic of this topic is:
205 # <a name="TWiki Installation Error"></a> TWiki Installation Error
207 Incorrect format of searchformat template (missing sections? There should be 4 %SPLIT% tags)
209 ### <a name="Solution 3: Using IF statement"></a> Solution 3: Using IF statement
211 This pattern can be rewritten using `%IF%`, removing the dependency on SpreadSheetPlugin:
213 %SEARCH{ "^%BASETOPIC%$" web="%BASEWEB%" scope="topic" type="regex" nonoise="on" format="$percntIF{$quot$parent$quot then=$quot * $parent$quot else=$quot<nop>$quot}$percnt" }%
215 #### <a name="Test Case"></a> Test Case
217 The parent topic of this topic is: %BR%
219 # <a name="TWiki Installation Error"></a> TWiki Installation Error
221 Incorrect format of searchformat template (missing sections? There should be 4 %SPLIT% tags)
223 <a name="SearchTopicChildren"></a>
225 ## <a name="Pattern 5: Show all Children of"></a><a name="Pattern 5: Show all Children of "></a> Pattern 5: Show all Children of a given topic
227 ### <a name="Problem"></a> Problem
229 How to get to the list of all children of the current topic to display on the page?
231 ### <a name="Solution"></a> Solution
233 The parent information is stored in the META:TOPICPARENT meta data. Do a SEARCH to find all topic parent meta data pointing to the current topic:
236 %SEARCH{ "META\:TOPICPARENT.*\"%TOPIC%\"" type="regex" nonoise="on" format="[[$topic]]" separator=", " }%
238 **_Note:_** Replace `%TOPIC%` with `%BASETOPIC%` if you put this SEARCH into the skin or a sidebar.
240 <a name="SearchPublicWebsList"></a>
242 ## <a name="Pattern 6: Search and display th"></a> Pattern 6: Search and display the home topics of public webs in a list
244 ### <a name="Problem"></a> Problem
246 How to find and display public webs in a drop down list box.
248 ### <a name="Solution"></a> Solution
250 _Thanks to TWiki:Main.PeterThoeny for these solutions._
253 <select name="topic">
254 <option value="%TOPIC%">Select...</option>
255 %SEARCH{ "%HOMETOPIC%" scope="topic" web="all" topic="%HOMETOPIC%" format="<option value=\"$web.$topic\">$web</option>" separator=" " }%
257 <input type="submit" value="Go" />
260 ### <a name="Test case"></a> Test case
262 Public webs of TWiki.
264 <form><select name="topic"><option value="%TOPIC%">Select...</option>
265 <h1><a name="TWiki Installation Error"></a> TWiki Installation Error </h1>Incorrect format of searchformat template (missing sections? There should be 4 %SPLIT% tags) </select> <input type="submit" value="Go" /></form>
267 %T% For private webs, or any other webs you wish to exclude from the display, use "on" for the `Exclude web from a web="all" search` setting in the relevant web's WebPreferences topic.
269 ### <a name="Alternative solution"></a> Alternative solution
271 This result can also be accomplished with the %WEBLIST% variable.
273 <form><select name="topic"><option value="%TOPIC%">Select...</option>
275 </p></select> <input type="submit" value="Go" /></form>
277 <a name="SearchBulletList"></a>
279 ## <a name="Pattern 7: Create a select box w"></a> Pattern 7: Create a select box with values from a bullet list
281 ### <a name="Problem"></a> Problem
283 We have a topic with a bullet list with category names. In another topic we want to offer these values in a select box dropdown.
285 For example, CategoryList has:
292 ### <a name="Solution"></a> Solution
294 The following search pattern can be employed:
297 <option>Select category...</option>
298 %SEARCH{" *\s*.*?" topic="CategoryList" type="regex" multiple="on" casesensitive="on" nosummary="on" nosearch="on" noheader="on" nototal="on" format="<option>$pattern(.* \*\s*([^\n]*).*)</option>"}%
301 To render the bullet list as a comma-separated list, use the `separator` parameter:
303 %SEARCH{" *\s*.*?" topic="CategoryList" type="regex" multiple="on" casesensitive="on" nosummary="on" nosearch="on" noheader="on" nototal="on" separator="," format="$pattern(.* \*\s*([^\n]*).*)"}%
305 <a name="SearchNamedBulletList"></a>
307 ## <a name="Pattern 8: Extract a value from"></a><a name="Pattern 8: Extract a value from "></a> Pattern 8: Extract a value from a named bullet list item
309 ### <a name="Problem"></a> Problem
311 Display the user name in the user's topic title
313 ### <a name="Solution"></a> Solution
315 Search for the `Name:` entry.
317 %SEARCH{" * [N]ame: " topic="%TOPIC%" type="regex" casesensitive="on" nosummary="on" nosearch="on" noheader="on" nototal="on" format="---+!! $pattern(.* \* Name: ([^\n]*).*)"}%
319 ### <a name="Test case"></a> Test case
321 To create a test case, we will put a name entry here:
327 # <a name="TWiki Installation Error"></a> TWiki Installation Error
329 Incorrect format of searchformat template (missing sections? There should be 4 %SPLIT% tags)
331 <a name="SearchMetaData"></a>
333 ## <a name="Pattern 9: Search for Form and M"></a> Pattern 9: Search for Form and Meta data: explained
335 ### <a name="Problem"></a> Problem
337 Below is an example of a search that searches form data. The questions are:
339 - why is this searching the metadata, shouldn't it just search the text?
340 - what is the meaning of the `td..td` in the search expression?
342 %SEARCH{ "[S]tatus.*(td..td|value\=).*[W]aiting" casesensitive="on" type="regex"
343 nosearch="on" nototal="on" format="| [[$topic]]<br /> ($date - $rev -
344 [[%SCRIPTURLPATH{rdiff}%/$web/$topic][Diffs]]) |"}%
346 ### <a name="Solution"></a> Solution
348 %SEARCH depends on grep, and grep searches the whole file, including the meta data.
350 An example meta data form field is:
352 %META:FIELD{name="OperatingSystem" title="OperatingSystem" value="OsWin"}%
354 So a search for a form field could look like:
356 %SEARCH{ "[O]peratingSystem.*value\=.*[O]sWin" type="regex" ... }%
358 - Using square brackets is a trick to avoid a hit on the topic doing the search.
359 - The `.*` indicate that there can be any number of any character between `OperatingSystem` and `value` in the (whole) file
361 Now the original file format of the category table (the predecessor of the TWiki forms) looks like this:
363 <td valign="top" align="right"> OperatingSystem: </td><td> OsWin </td>
365 The following search finds topics in the old and new format:
367 %SEARCH{ "[O]peratingSystem.*(td..td|value\=).*[O]sWin" type="regex" ... }%
369 The `td..td` matches `td<>td`; a simple search on `"[O]peratingSystem.*[O]sWin"` could find a hit in the topic text by coincidence.
371 A simple `%SEARCH{ "[O]peratingSystem.*value\=.*[O]sWin" ...}%` search is sufficient if you do not have topics in the old format.
373 <a name="MovedTopics"></a>
375 ## <a name="Pattern 10: Search all topics th"></a> Pattern 10: Search all topics that have been moved
377 ### <a name="Problem"></a> Problem
379 How would I go about listing all moved topics ?
381 ### <a name="Solution"></a> Solution
383 Search for the META:TOPICMOVED meta data. Type this:
385 `Moved topics: %SEARCH{ "%META\:TOPICMOVED" type="regex" format="$topic, " nosearch="on" noheader="on" nosummary="on" }%`
387 to get this (limited to 10 results):
391 # TWiki Installation Error
393 Incorrect format of searchformat template (missing sections? There should be 4 %SPLIT% tags)
395 **_Related Topics:_** [[UserDocumentationCategory]], [[SearchHelp]], [[TWikiVariables#VarSEARCH]], [[FormattedSearch]], [[RegularExpression]]
397 -- **_Contributors:_** TWiki:Main.AntonAylward, TWiki:Main.ArthurClemens, TWiki:Main.JosMaccabiani, TWiki:Main.PeterThoeny, TWiki:Main.SueLocke