{"id":7,"date":"2022-08-09T17:36:00","date_gmt":"2022-08-09T16:36:00","guid":{"rendered":"https:\/\/blogs-staging.imperial.ac.uk\/technology-office\/?p=7"},"modified":"2022-08-09T17:36:00","modified_gmt":"2022-08-09T16:36:00","slug":"demystify-regular-expressions","status":"publish","type":"post","link":"https:\/\/blogs-staging.imperial.ac.uk\/technology-office\/2022\/08\/09\/demystify-regular-expressions\/","title":{"rendered":"Demystify regular expressions"},"content":{"rendered":"

Have you ever filled a form where you got an “invalid email address” error? These alerts are commonly built using regular expressions<\/strong>. They evaluate the field value against a set of rules and report if the value complies with them. Regular expressions allow much more than this simple example.<\/p>\n

I was introduced to regular expressions (RegEx <\/strong>for short) in 2009 when studying Finite Automaton. A Finite Automaton (aka Finite State Machine (FSM)) is a state machine that accepts or rejects input based on a set of rules. The example I found most interesting is the concept of an arcade Galaga machine that would, in the 80s, accept a coin as the input. It would return you a credit to spend on the game if the coin was 10p. If the coin was another coin, it would reject the input and return the coin back to you.<\/p>\n

One of the most know automaton machines was Alan Turing<\/a>‘s Turing machine<\/a> which was used to break the Enigma machine in the 1940s. Regular expressions came shortly after in 1951 by mathematician Stephen Kleene<\/a>, and its first know application was made in 1961 by Ken Thomson<\/a> on his Unix text editor “ed”.<\/p>\n

How does RegEx work?<\/h2>\n

RegEx uses a search pattern that can match, replace and manage text. It can significantly simplify going through many text files, logs, spreadsheets, etc. I love using it in my code when I need to match, select and manipulate objects. Very useful in XPath, python and PowerShell scripts.<\/p>\n

RegEx syntax summary<\/h2>\n
\r\nLiteral characters:<\/strong> Except for special characters all characters match their literal characters. E.g. the regex for \"A\" matches the string \"A\", \"9\" matches the number 9, \"=\" matches \"=\", etc.
\r\nSpecial Characters:<\/strong> ., +, *, ?, ^, $, (, ), [, ], {, }, |, \\.
\r\nEscape sequence:<\/strong> If you need to match any of the special characters, you need to add an escape character \"\\\". E.g. \"\\\\\" will match \"\\\" and \"\\[\" will match \"[\"
\r\nOR operator \"|\":<\/strong> you can try to match multiple strings like \"Robert|Rob\" this will match any of the words \"Robert\" or \"Rob\"
\r\nMetacharacters:<\/strong>
\r\n    . will match any single character <\/span>
\r\n    \\d will match any single digit<\/span>
\r\n    \\D will match any NON single digit<\/span>
\r\n    \\w will match any single lower\/upper case A to Z and 0 to 9<\/span>
\r\n    \\W will match any single NON lower\/upper case A to Z and 0 to 9<\/span>
\r\n    \\s will match any single space<\/span>
\r\n    \\S will match any single NON space<\/span>
\r\nPosition Anchors:<\/strong>
\r\n    ^ start of line<\/span>
\r\n    $ end of line<\/span>
\r\n    \\r carriage return<\/span>
\r\n    \\n line feed<\/span>
\r\nLaziness matching:<\/strong>
\r\n    . any single character<\/span>
\r\n    * multiple times<\/span>
\r\n    + one or more times<\/span>
\r\n    ? zero or more times<\/span>
\r\n    {,m} between 0 and m times<\/span>
\r\n    {n,} more than n times<\/span>
\r\n    {n,m} between n and m times<\/span>
\r\nCapturing groups:<\/strong>
\r\n    () will capture a back reference to the regex within the parentheses. You can access these by referencing them as E.G. \\1, $1, {1}, etc, depending on the language\/tool used.<\/span>
\r\n<\/pre>\n
Powershell:<\/strong><\/p>\n
\n
\r\n  # How to show any occurrence of the word error in one folder<\/span>            
  Get-ChildItem<\/span> -Path<\/span> $logFolder<\/span> |<\/span> Select-String<\/span> -Pattern<\/span> 'Error'<\/span><\/code><\/pre>\n<\/div>\n
Python:<\/strong><\/p>\n
\n
\r\n  # How to click on an a HTML object using Python and Selenium<\/span>            
  driver.find_element(By.XPATH,<\/span>\"\/\/a[i[contains(@value,'objName')]]\"<\/span> ).click()<\/span><\/code><\/pre>\n<\/div>\n
You can try regular expressions straight from your browser. I found it useful when automating web checks or completing web scraping activities. If you use Chrome, press ctrl_shift+i (F12 in most browsers) to open the developer tools. In the “console” tab, you should be able to identify if an element exists by typing \"$x('expression')\"<\/code> and Chrome will let you know if the element exists.
\n\"\"<\/p>\n
This can also be tested within the “elements tab”:
\n\"Elements<\/p>\n
I also tend to use it with text editors (Notepad++, Geany, etc.) for quick text cleanups.  As I was writing this, I wanted to add the entire list of rooms available on a form list box existing in one of the ICT events systems. I could have typed them one by one, but that is not a good way to do it if you have more than a handful of entries. I’ve copied the HTML element I wanted and used Notepad++. <\/p>\n
I wanted the room names just like the ones highlighted below:
\n\"\"<\/p>\n
Using the “Replace” option under the “Search” menu on NotePad++ and having the “Regular Expression” option ticked, I typed ^.*([0-9a-zA-Z. ]*)[\\t ]*+<\/code> and replaced it with \\1<\/code>:
\n\"\"
\n^<\/span>.*<\/span><option<\/span>.*<\/span>><\/span>([0-9a-zA-Z. ]*)<\/span>[\\t ]*<\/span><\/.*><\/span><\/code> is signalling to the search and replace function that I wanted to find:<\/p>\n
    \n
  1. start my search from the beginning of the line<\/span><\/li>\n
  2. has any number of caracters<\/span> before it encounters <option<\/span><\/li>\n
  3. has any number of caracters<\/span> before it encounters ><\/span><\/li>\n
  4. This part is a group because it is wrapped with (). Looks for any word of any length composed of numbers, lower\/upper case characters from A to Z, dots and spaces.<\/span><\/li>\n
  5. has any number of spaces<\/span><\/li>\n
  6. has any number of characters between <> <\/li>\n<\/ol>\n
    The replace box signals that the entire word match should be replaced with the first group encountered in the match. The result can be seen as:
    \n\"Console<\/p>\n
    Conclusion<\/h2>\n
    There are hundreds of sites explaining RegEx but I’ll point you to the reputable Microsoft page: https:\/\/docs.microsoft.com\/en-us\/dotnet\/standard\/base-types\/regular-expression-language-quick-reference<\/a> for reference.<\/p>\n
    The potential for using RegEx is immense. Have you used it before? Any good examples you can share?<\/p>\n","protected":false},"excerpt":{"rendered":"
    Have you ever filled a form where you got an “invalid email address” error? These alerts are commonly built using regular expressions. They evaluate the field value against a set of rules and report if the value complies with them. Regular expressions allow much more than this simple example. I was introduced to regular expressions […]<\/p>\n","protected":false},"author":1672,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[320486],"tags":[],"class_list":["post-7","post","type-post","status-publish","format-standard","hentry","category-tools-and-tips"],"_links":{"self":[{"href":"https:\/\/blogs-staging.imperial.ac.uk\/technology-office\/wp-json\/wp\/v2\/posts\/7","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs-staging.imperial.ac.uk\/technology-office\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs-staging.imperial.ac.uk\/technology-office\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs-staging.imperial.ac.uk\/technology-office\/wp-json\/wp\/v2\/users\/1672"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs-staging.imperial.ac.uk\/technology-office\/wp-json\/wp\/v2\/comments?post=7"}],"version-history":[{"count":77,"href":"https:\/\/blogs-staging.imperial.ac.uk\/technology-office\/wp-json\/wp\/v2\/posts\/7\/revisions"}],"predecessor-version":[{"id":93,"href":"https:\/\/blogs-staging.imperial.ac.uk\/technology-office\/wp-json\/wp\/v2\/posts\/7\/revisions\/93"}],"wp:attachment":[{"href":"https:\/\/blogs-staging.imperial.ac.uk\/technology-office\/wp-json\/wp\/v2\/media?parent=7"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs-staging.imperial.ac.uk\/technology-office\/wp-json\/wp\/v2\/categories?post=7"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs-staging.imperial.ac.uk\/technology-office\/wp-json\/wp\/v2\/tags?post=7"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}