Save dynamic list to csv

web-scraping

#1

Hello, I recently discover Kantu this amazing tool, and have put it to good use for several of my tedious manual tasks. I’m currently stuck on a huge macro, I have most of it working with the exception of saving the contents of a dynamic list to a csv file. I have a basic understanding of how to save values to a csv file and write it to the local storage. The part I’m stuck on is how retrieve the values from the dynamic list. The values are dynamic as well as the number of items in the list. the list can have from 1 to 50 items. The values in the list are always 6 characters long, and a combination of numeric and alpha numeric values. Here is a sample of what the html looks like:

<td>

<select name="acctNumber" id="acctNumber" style="font-size: 7.5pt; visibility: visible;" onblur="" onchange="" onfocus="" size="1" onclick="">

<option value="A1234">A1234</option>

<option value="2B345">23B45</option>

<option value="345C6">345C6</option>

</select>

</td>

I have tried using storeValue, RegEx, StoreEval, and a few other functions to no avail. I’m certain this is possible I’m just stuck. I have considered saving the entire webpage to the local storage and parsing it using vba, but I would prefer to get it working in Kantu. Any help would be greatly appreciated.


#2

What about using storeText?

If I run this macro:

{
  "Name": "select web scraping",
  "CreationDate": "2019-3-15",
  "Commands": [
    {
      "Command": "open",
      "Target": "https://a9t9.com/contact",
      "Value": ""
    },
    {
      "Command": "storeText",
      "Target": "id=WhatSoftware",
      "Value": "v1"
    },
    {
      "Command": "echo",
      "Target": "v1=${v1}",
      "Value": ""
    }
  ]
}

I get this output:

v1=--- Please select Software ---
Kantu for Chrome/Firefox
OCR.space Free Plan
OCR.space PRO/PROPDF Plan [Priority Support]
SeeShell Community Editions
SeeShell PRO Editions [Priority Support]
Other...

Are these values you need?

See also Web Scraping with Selenium IDE


#3

Thank you Ulrich, I really appreciate you taking to time to respond. Yes, that is exactly the output I’m looking to get from my acctNumber list. I see what you did and it looks like it should work for my application, however I must be missing something?

I tried the following:
{
“Command”: “open”,
“Target”: “https://actual.webaddress.goes.here.com/Login.action”,
“Value”: “”
},
{
“Command”: “storeText”,
“Target”: “id=acctNumber”,
“Value”: “v1”
},
{
“Command”: “echo”,
“Target”: “v1=${v1}”,
“Value”: “”
}
]
}

and it returns:

  • [info]

Executing: | storeText | id=acctNumber | v1 |

  • [info]

Executing: | echo | v1=${v1} | |

  • [echo]

v1=

Any idea what I did wrong?


#4

if I use StoreValue

{
  "Command": "storeValue",
  "Target": "id=acctNumber",
  "Value": "v1"
},
{
  "Command": "echo",
  "Target": "v1=${v1}",
  "Value": ""
}

]
}

it returns a single value, see below:

  • [info]

Executing: | storeValue | id=acctNumber | v1 |

  • [info]

Executing: | echo | v1=${v1} | |

  • [echo]

v1=XXX33X


#5

I guess something is different with your website’s HTML, but I am not sure what.

Have you tried using sourceExtract to get the HTML source code directly?


#6

I’m trying to use sourceExtract but I’m not using it correctly. would you suggest using it with RegEx? If you would be so kind as to provide sample usage similar to your example for StoreValue, I would really appreciate it.