Regex matching

[pokrakam]

Suggestion: Could regex be incorporated in the matcher?

One immediate use case in ABAP is an end of line comment " is OK and " in strings need to be kept, but a line starting with " or * should be skipped.

* Now we will loop through the table
loop at sometable reference into data(row).
  if row->somefield = 'foo'. 
    "Exclude these
    continue.
  endif.
  write / |Row { sy-tabix } contains "{ row->anotherfield }"|. 
endloop. "end-of line comments can be deleted or not...

Should go to

loop at sometable reference into data(row).
  if row->somefield = 'foo'. 
    continue.
  endif.
  write / |Row { sy-tabix } contains "{ row->anotherfield }"|. 
endloop. "end-of line comments can be deleted or not...

Easy to do with regex, but I can't see a way according to the current documentation.

[iHiD]

I think you can do this with the basic mode syntax: https://github.com/exercism/snippet-extractor/blob/main/docs/basic.md

But if not, you can definitely do it with extended mode: https://github.com/exercism/snippet-extractor/blob/main/docs/extended.md

@joshiraez Could you lend a hand pls :)

[pokrakam]

Thanks, I'm not sure this is the case. The extended mode syntax itself is not compatible with regex. I can enter * in extended mode to match a * and nothing else, whereas in regex it would match anything. I did try matching ^" to indicate beginning of line but it did not work.
I do notice the extended mode has some regex-ish syntax like the +, but I don't believe it's proper regex.

[joshiraez]

I think you are confused on what extended mode does.

It's a design decision to not use regex and I dont foresee it changing because of two reasons

• UX: a lot of people hate regex and it gets cumbersome for very basic cases. Even very simple checks can quickly go awry when you try to do global searches
• Efficiency: regex starts to be REALLY slow the moment you introduce lookouts. For something that was going to run over a lot of files, regex seemed like it could be a pipe factor

I understand that for regex users, changing to this syntsx can be frustrating, specially because it brings a subset of regex features and it can be hard to adjust, but it has its reasons

If you are interested, it uses a trie parser behind the doors using as basis the syntax that the snippet refactor used to have.

Now for your problem. You'd need the extended syntax (so it looks up the whole file) but it would suffice with two lines

!e
*
"

Those 2 lines without modifiers mean "any line that starts with * or " must be skipped". It ignores tabs, so don't worry about that.

In regex, they are similar to

[\*\"].*\n

I think this suffices what you are asking for. As those in the end of line shouldnt be skipped, dont they?

In any case, please try it and we can iterate from there

[pokrakam]

Thanks @joshiraez for the explanation.

Hmm, ok I'm having real trouble doing a halfway useful ABAP extractor with either syntax. The comments matching was just dipping my toes in, the bigger issue I've moved onto is a multiline match:

ABAP uses class ... definition and class ... implementation sections, both defined with the same keywords. All snippets look the same because we only see the definitions that are provided as part of the exercise.

Therefore removing the definition section is way more important than removing comments. But after spending a good couple of hours on learning all the ins and outs (and a little Ruby along the way 🙂), I can't get it to work.

Example:

class foo definition. 
  public section.
    "! AbapDoc comment 
    methods run.
  private section.
    data stuff type standard table of string.
endclass.                    
                                
class foo implementation. 
  method run.
    " Some comment
    write 'Hello world!'.
  endmethod.
endclass.

Desired result:

class foo implementation. 
  method run.
    " Some comment   <-- Would be nice to remove but OK if we can't
    write 'Hello world!'.
  endmethod.
endclass.

The problem is that class-->>endclass\p will remove both sections.
A regex variant such as class (.+)definition-->>endclass\p would do the job.

I know we're drifting a little off my original topic, happy to move this to a new issue if preferred.
I've also uploaded my current test in a draft PR #42.

Example of the APAP preview snippets at https://exercism.org/tracks/abap/exercises/two-fer/solutions , which shows exactly the reason I wanted to create this.

[joshiraez]

I'd appreciate if you could give me an example with all the cases you need to be removed and I'll be happy to work on it and explain it later to you in case you need anything more. If you can send me a original/extractwd to build an unit test and wprk from there it'd be amazing El jue., 31 mar. 2022 22:58, Mike Pokraka ***@***.***> escribió:

Thanks @joshiraez <https://github.com/joshiraez> for the explanation. Hmm, ok I'm having real trouble doing a halfway useful ABAP extractor with either syntax. The issue is ABAP uses a class definition and implementation section, both defined with the same keywords. The definitions are all that we see on any of the solutions snippets, so all snippets look the same. Therefore removing the definition section is way more important than comments. But after spending a good couple of hours on it I can't get it to work. Here's an example: class foo definition. public section. "! AbapDoc comment methods run. private section. data stuff type standard table of string.endclass. class foo implementation. method run. " Some comment write 'Hello world!'. endmethod.endclass. Desired result: class foo implementation. method run. " Some comment <-- Would be nice to remove but OK if we can't write 'Hello world!'. endmethod.endclass. The problem is that class-->>endclass\p will remove *both* sections. A regex variant such as class (.+)definition-->>endclass\p would do the job. I know we're drifting a little off my original topic, happy to move this to a new issue if preferred. I've also uploaded my current test in a draft PR #42 <#42> if you wanted to have a look. Example of the APAP preview snippets at https://exercism.org/tracks/abap/exercises/two-fer/solutions , which shows exactly the reason I wanted to create this. [image: image] <https://user-images.githubusercontent.com/25539927/161144712-99023f7e-1cc8-4cb9-a6c4-abfc7e344f5d.png> — Reply to this email directly, view it on GitHub <#41 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEEIPLHLIKBFU3MI6UTFD5DVCYGYJANCNFSM5SGCHIAQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

Edit: I didnt see the draft. That will do nicely. Thank you.

[joshiraez]

Btw, I dont remember if i did it in the end but i had the idea you could do something like

Class -->> definition. -->> endclass

If thats not implemented ill have to extend the extractor a little, but it should be no biggie.

[pokrakam]

Thanks, I did try the chained variant and it didn't work. I even tried a real dodgy hack:

class-->>defin\pj
ition.-->>endclass\p

Not sure why that won't work, but it would fall apart if "defin" is part of a class name.

[joshiraez]

Because it stops at the first match Ill get on with implementing the multi arrow then. Or see any way to support this. El jue., 31 mar. 2022 23:47, Mike Pokraka ***@***.***> escribió:

…

Thanks, I did try the chained variant and it didn't work. I even tried a real dodgy hack: class-->>defin\pj ition.-->>endclass\p Not sure why that won't work, but it would fall apart if "defin" is part of a class name. — Reply to this email directly, view it on GitHub <#41 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEEIPLH26UI2XKNYMVVV3XTVCYMQBANCNFSM5SGCHIAQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[SleeplessByte]

@joshiraez can we introduce ~ as "includes" ? then you could do ~ implementation

[joshiraez]

Mmm, can you put how the rule would be written with your syntax? I dont think the double arrow has good ux, but from what you said `class.~definition.~end.` seems even more confusing if you have to remember what differences are between arrow and this one. Meanwhile, `class.-->>definition.-->>end.` seems self explanatory. In any case, is not an easy a fix as I thought. Im going to spend a few days to do a small refactor on the trie parser. We can discuss the syntax here in the meanwhile. El vie., 1 abr. 2022 19:06, Derk-Jan Karrenbeld ***@***.***> escribió:

…

@joshiraez <https://github.com/joshiraez> can we introduce ~ as "includes" ? then you could do ~ implementation — Reply to this email directly, view it on GitHub <#41 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEEIPLENIIVHDJMRVTODPVDVC4UJDANCNFSM5SGCHIAQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[pokrakam]

I think a wildcard would be useful. Many apps that don't use regex still use * or ? as wildcards. I see * is already in use as a matching pattern (and ~ for that matter), but wouldn't be a big job to escape the existing ones - happy to do it.
So in my example it would be:
class*definition-->>endclass
This seems intuitive to me, * is a match on the same line and -->> is a multi-line match.

[joshiraez]

But in your example the wildcard is exactly the same as the arrow. Also, if you use * as wildcard you are then losing that character as a valid input character, when its one of the most common ways to comment stuff. I dont think we should copy regex syntax as I said before, just because its regex syntax El sáb., 2 abr. 2022 9:57, Mike Pokraka ***@***.***> escribió:

…

I think a wildcard would be useful. Many apps that don't use regex still use * or ? as wildcards. I see * is already in use (and ~ for that matter), but wouldn't be a big job to escape the existing ones - happy to do it. So in my example it would be: class*definition-->>endclass This seems intuitive to me. — Reply to this email directly, view it on GitHub <#41 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEEIPLFOLBPDJRRPZNWYIGDVC74W7ANCNFSM5SGCHIAQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[joshiraez]

I mean, if you are so keen on introducing regex, I can just add an argument to the extended mode to use regex instead of the trie parsers. I just dont think is a good design solution and it was one of the things we wanted to avoid from the start, but I feel like thats what you are pushing for...? What does everyone think about this? El sáb., 2 abr. 2022 11:29, Joshi Raez ***@***.***> escribió:

…

But in your example the wildcard is exactly the same as the arrow. Also, if you use * as wildcard you are then losing that character as a valid input character, when its one of the most common ways to comment stuff. I dont think we should copy regex syntax as I said before, just because its regex syntax El sáb., 2 abr. 2022 9:57, Mike Pokraka ***@***.***> escribió: > I think a wildcard would be useful. Many apps that don't use regex still > use * or ? as wildcards. I see * is already in use (and ~ for that > matter), but wouldn't be a big job to escape the existing ones - happy to > do it. > So in my example it would be: > class*definition-->>endclass > This seems intuitive to me. > > — > Reply to this email directly, view it on GitHub > <#41 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AEEIPLFOLBPDJRRPZNWYIGDVC74W7ANCNFSM5SGCHIAQ> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >