shell - need help in grep (BASH)

Question

I try to make a tool to get list of proxies and You have downloaded Index for one of the free proxy sites I use this :

wget http://free-proxy.cz/en/proxylist/country/all/https/ping/all

and outputs something like that :

<script type="text/javascript">document.write(Base64.decode("MTg1LjExNC4xMzcuMTQ="))</script></td><td style=""><span class="fport" style=''>12195</span></td><td><small>HTTPS</small></td><td class="left"><div style="padding-left:2px"><img src="/flags/blank.gif" class="flag flag-ua" alt="Ukraine" /> <a href="/en/proxylist/country/UA/all/ping/all">Ukraine</a></div></td><td class="small"><small></small></td><td class="small"><small></small></td><td class="small"><small>High anonymity</small></td><td> <i class="icon-black icon-question-sign"></i></td><td><small>2.4%</small><div class="progress"><div class="fill" style="width:4%;background-color:red;"></div></div></td><td><div style="padding-left:5px"><small>649 ms</small> <div class="progress"><div class="fill" style="width:94%;background-color:#A5DA74;;"></div></div></div></td><td><small>8 hours ago</small></td></tr><tr><td style="text-align:center" class="left"><script type="text/javascript">document.write(Base64.decode("MTYxLjk3LjEzOC4yMzg="))</script></td><td style=""><span class="fport" style=''>3128</span></td><td>

As you can see the IP is encrypted in base64 and port is normal

I try to grep base64 codes first and this is work ↓

echo (outputs) | grep -Eo '("[A-Za-z0-9]{12,30}[=]{0,2}")' | cut -d '"' -f2

and I try this code to get ports ↓

echo (output) | grep -Eo "(class="fport" style=''>[0-9]{1,9})"  | cut -d '>' -f2

how can I mix it and make it like that

(base64 code):(port)

and after that I wanna decrypt the base64 code and make it look like :

IP:PORT

Answer 1

1st step

base64 is not an encryption, but an encoding. If you are working on Linux or other Unix-variants, the command base64, base64-encoder/decoder, will be pre-installed. If not, it will be easily installed with your OS-dependent package manager. Then please try to execute:

base64 -d <<< "MTg1LjExNC4xMzcuMTQ="

It will output:

185.114.137.14

2nd step

Then we can combine the base64 decoder with your command pipeline. The problem is base64-coding ignores newlines and we need to process the result of the pipeline line by line. Assuming the variable $output holds the output of the wget command, please try:

while IFS= read -r line; do
    base64 -d <<< "$line"
    echo
done < <(echo "$output" | grep -Eo '("[A-Za-z0-9]{12,30}[=]{0,2}")' | cut -d '"' -f2)

It will print something like:

185.114.137.14
161.97.138.238

The <(command) notation is a process substitution and the output of echo .. grep .. cut pipeline is fed to the while loop via stdin and the while loop processes the base64-encoded strings line by line.

3rd step

Now we want to merge the IPs and PORTs in a format of IP:PORT. We can make use of paste command. The final script will be:

paste -d ":" <(
while IFS= read -r line; do
    base64 -d <<< "$line"
    echo
done < <(echo "$output" | grep -Eo '("[A-Za-z0-9]{12,30}[=]{0,2}")' | cut -d '"' -f2)
) 
<(echo "$output" | grep -Eo "(class="fport" style=''>[0-9]{1,9})"  | cut -d '>' -f2)

Output:

185.114.137.14:12195
161.97.138.238:3128

The paste command takes filenames as arguments. Here we make use of process substitution again in a manner as: paste <(command1) <(command2) which saves to create temporary files.