You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Closes#232 . Kept all checklists instead of updating to challenges as this is an advanced lesson and does not expect learners to be able to write their own regular expressions.
Copy file name to clipboardExpand all lines: episodes/10-regular-expressions.md
+19-19Lines changed: 19 additions & 19 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -26,24 +26,24 @@ exercises: 20
26
26
27
27
Regular expressions, often referred to as regex, are a tool you can use to match, capture and manipulate data across your MARC record file. Regular expressions are comprised of a sequence of literal characters and metacharacters which are formulated to match a particular pattern found in your data that you want to manipulate. A detailed introduction to working with regular expressions can be found in the [Library Carpentry lesson Introduction to Working with Data (Regular Expressions)](https://librarycarpentry.org/lc-data-intro/index.html). In this lesson we will focus on the application of regex in the MarcEditor.
28
28
29
-
MarcEdit uses the .NET regular expression syntax. Visit the .Net[Regular Expression Language - Quick Reference](https://docs.microsoft.com/en-us/dotnet/standard/base-types/regular-expression-language-quick-reference) for a breakdown of the syntax used for regex metacharacters in MarcEdit. You can also download the [.Net Regex metacharacter reference sheet](https://download.microsoft.com/download/D/2/4/D240EBF6-A9BA-4E4F-A63F-AEB6DA0B921C/Regular%20expressions%20quick%20reference.pdf) to support the creation of your own regular expressions in MarcEdit.
29
+
MarcEdit uses the .NET regular expression syntax. Visit the .NET[Regular Expression Language - Quick Reference](https://docs.microsoft.com/en-us/dotnet/standard/base-types/regular-expression-language-quick-reference) for a breakdown of the syntax used for regex metacharacters in MarcEdit. You can also download the [.NET Regex metacharacter reference sheet](https://download.microsoft.com/download/D/2/4/D240EBF6-A9BA-4E4F-A63F-AEB6DA0B921C/Regular%20expressions%20quick%20reference.pdf) to support the creation of your own regular expressions in MarcEdit.
30
30
31
31
Many MarcEdit functions support the use of regular expressions for data selection and manipulation. They include: Find and Replace, Edit Field, Copy Field, Swap Field, Build New Field, Delete Field, Record Validation, and both the Extract and Delete Selected Records tool. The intent of this lesson is to demonstrate where and how you can leverage regular expressions to work with your MARC data.
32
32
33
-
###Edit Field Data
33
+
## Edit Field Data
34
34
35
35
With regular expressions you can easily locate variations in your data and replace them with a single value. In our file the OCLC identifier in the 035 field uses the prefixes `on`, `ocn`, and `ocm`, but our library formats the OCLC with the `(OCoLC)` prefix. Instead of running three Edit Field Data functions to update these prefixes, we can use a regular expression in the Edit Field Data tool to replace these prefixes with the `(OCoLC)` prefix in a single edit.
5. Verify your results look as expected, close preview tab and then click Process
44
+
3. Check Use Regular Expressions box.
45
+
4. Click drop down menu → Preview Results.
46
+
5. Verify your results look as expected, close preview tab and then click Process.
47
47
In this regular expression we use the pipe (`|`) metacharacter to specify that we are looking for the string `on` OR `ocn` OR `ocm`. We then replace the captured string with `(OCoLC)`.
48
48
49
49
@@ -62,30 +62,30 @@ Reviewing your dataset before employing regular expressions is a good best pract
The Find and Replace tool is another useful way to identify and manipulate data in the MarcEditor with the support of regular expressions. In our file the local call numbers in the `090` field are formatted without a space between the class and subclass (ex. `LD1780` and not `LD 1780`). To update the call number to follow our local policy we need to isolate the components of the call number so we can add a space between them. We can do this with a regular expression by employing groups.
68
68
69
69
::::::::::::::::::::::::::::::::::::::: checklist
70
70
71
71
### Add a space between 099 class and subclass
72
72
73
-
1. Select Edit → Find
74
-
2. Enter `=099` in the Find box and click Find All
73
+
1. Select Edit → Find.
74
+
2. Enter `=099` in the Find box and click Find All.
75
75
Review the list of `099` fields, what observations can you make about the call numbers in this field?
76
76
Review shows us that not all call numbers begin with letters. We do not want to add a space to these call numbers and need to account for this in our regular expression.
77
-
3. In the Find window, click Replace to bring up the Replace Text functions
78
-
4. In the Find box enter the regular expression `(=099 \\\\\$a[A-Z]+)(\d.*)`
77
+
3. In the Find window, click Replace to bring up the Replace Text functions.
78
+
4. In the Find box enter the regular expression `(=099 \\\\\$a[A-Z]+)(\d.*)`.
79
79
This regular expression uses two sets of `()` to capture groups and then manipulate them.
80
80
81
81
- Broken down, the first group `(=099 \\\\\$a[A-Z]+)` will match the literal string `=099 \\$a` (the additional backslashes are used to escape the regex metacharacters `\` and `$` so that they will be read as literals) followed by any single capital letter in the range A to Z (`[A-Z]`) one or more times (`+`). By specifying that the regular expression must locate a capital letter at the start of the call number, the regular expression will not edit the call numbers that begin with a digit.
82
-
- The second group `(\d.*)` will match any digit (`\d`) followed by any character (`.`) zero or more times (`*`)
82
+
- The second group `(\d.*)` will match any digit (`\d`) followed by any character (`.`) zero or more times (`*`).
83
83
84
-
5. In the Replace box enter `$1 $2`
84
+
5. In the Replace box enter `$1 $2`.
85
85
This regular expression refers back to the groups we defined and captured in the Find box. `$1` refers to the group defined in the first set of brackets, and `$2` refers to the group defined in the second set of brackets. If we had defined additional groups they would be referred to chronologically as `$3`, `$4` etc. The regular expression `$1 $2` will output the contents of the two captured groups with a space between them.
86
-
6. Check the Use regular expressions box
87
-
7. Click drop down menu → Preview Results
88
-
8. Verify your results look as expected, close preview tab and then click Process
86
+
6. Check the Use regular expressions box.
87
+
7. Click drop down menu → Preview Results.
88
+
8. Verify your results look as expected, close preview tab and then click Process.
The Select Records for Edit tool, introduced in lesson 6, allows you to isolate and then work with a subset of your records. Regular expressions can be a powerful tool to help you isolate the subset of records you want to work with.
105
105
@@ -138,13 +138,13 @@ When using the Select Records for Edit tool you can use the Invert Selections op
138
138
139
139
Now that we've isolated our continuing resource records, we can add a field for the electronic journals genre form (`655 \4$aElectronic journals`) to the electronic records in our subset.
140
140
141
-
1. From the top level menu select Tools → Add/Delete Fields
141
+
1. From the top level menu select Tools → Add/Delete Fields.
142
142
2. In the Batch Editing window, enter 655 in the Field box and enter `\4$aElectronic journals` in the Field Data Box. Under General Options select Use Regular Expression, and under Add Field Options select Add Field If Present/NOT Present.
143
143
3. To add our 655 field to electronic records only, we need to specify what should be present in the record for the new field to be added.
144
144
What are some fields and values we might look at to determine format?
145
145
For this example we are going to use a regular expression to look for the presence of an 856 link or a the carrier type online resource. Enter the regular expression `=856|=338.+online resource` in the Find What box.
146
146
147
-
- This regular expression looks for the string `=856` OR the string `=338` followed by any character (`.`) one or more times (`+`), followed by the string online resource
147
+
- This regular expression looks for the string `=856` OR the string `=338` followed by any character (`.`) one or more times (`+`), followed by the string online resource.
148
148
149
149
4. Click Add Field. Review your data, were the new fields added correctly?
0 commit comments