LSP Regex & Makelist Discussion.

m5Germany

Active member
Mar 20, 2014
281
87
28
Greetings,
Would like some help on/with https://nflstream.io/nfl-network-live/stream-1. It has Post w/ cookie (needed).
Also, https://www.tvply.me/peercheck
Final Goal is to write a makelist for: https://nflstream.io/
I tried writing the Post but no success. Thanks, m5
 

gujal

Developer
Dec 26, 2013
564
311
63
The post can be done but the returning page has lot of javascript to process to get the actual `h.u.n.t.e.r` obfuscated code. Once you get that code you can use the jsunhunt and get the streamurl. but getting the obfuscated code is a complex task
The code below pulls the page with the correct post. The page now needs to be processed for all the javascript in it and deobfuscate before you can regex the playUrl
Code:
<item>
<title>NFLnet IO</title>
<link>https:$doregex[getUrl]</link>
<thumbnail></thumbnail>
<regex>
<name>getUrl</name>
<expres>playUrl\s*:\s*'([^']+)</expres>
<page>https://www.tvply.me/sdembed?v=$doregex[postparam]</page>
<rawpost>pid=$doregex[pid]&amp;ptxt=$doregex[postdata]</rawpost>
<referer>https://nflstream.io/</referer>
<origin>https://nflstream.io</origin>
</regex>
<regex>
<name>pid</name>
<expres>pid\s*=\s*(\d+)</expres>
<page>https://nflstream.io/nfl-network-live/stream-1</page>
<referer>https://nflstream.io/</referer>
</regex>
<regex>
<name>postdata</name>
<expres>pdettxt\s*=\s*"([^"]+)</expres>
<page>https://nflstream.io/nfl-network-live/stream-1</page>
<referer>https://nflstream.io/</referer>
</regex>
<regex>
<name>postparam</name>
<expres>zmid\s*=\s*"([^"]+)</expres>
<page>https://nflstream.io/nfl-network-live/stream-1</page>
<referer>https://nflstream.io/</referer>
</regex>
<thumbnail></thumbnail>
</item>
 
  • Like
Reactions: m5Germany

m5Germany

Active member
Mar 20, 2014
281
87
28
Gujal,
Glad to see you again. Still can't get this Post thing right.
I read whatever I can but without real Code examples, I get lost.
I understand all you have written, Thank You, and did similar w/ a
prior example you had written for me and tried to adapt from there.

<regex>
<name>getUrl</name>
<expres>link":\s*"([^"]+)</expres>
<page>h tps://www.stream2watch.one/calls/get/source</page>
<referer>h tps://www.stream2watch.one/</referer>
<origin>h tps://www.stream2watch.one</origin>
<x-req>XMLHttpRequest</x-req>
<post>h:$doregex[data]</post>
</regex>
<regex>
<name>data</name>
<expres>data-sh="([^"]+)</expres>
<page>h tps://www.stream2watch.one/streaming-television/eurosport-stream</page>
<referer>h tps://www.stream2watch.one/</referer>
</regex>

Anyway, Thanks for explaining. I do not have enough knowledge to move forward with site.

Couple of questions:
You know I use Fiddler - Is there something else you use that would help me see more?
Any chance I can get your opinion on cfscrape.create_scraper() from Post #1,420

If not, I understand.
Thanks, m5
 

gujal

Developer
Dec 26, 2013
564
311
63
cfscrape doesnt work anymore ever since cloudflare changed the mechanisms last year. So unfortunately cant help there. There is no working python code to bypass cloudflare currently. There is another project called cloudscraper which also is not working for most cloudflare sites
 
  • Like
Reactions: m5Germany

arslion84

New member
Dec 31, 2015
7
0
1
Hi all

can anybody help me to fix

XML:
<item><title>[COLOR violet]• [B]Süperlig[/B][/COLOR] Maç Özetler</title><link>$doregex[makelist]</link><regex><name>makelist</name><listrepeat><![CDATA[<title>[makelist.param1]</title><link>$doregex[video]</link><thumbnail>NA</thumbnail>]]></listrepeat><expres>data-title='(.*?)'.*data-video='(.*?)'</expres><page>https://beinsports.com.tr/mac-ozetleri-goller/super-lig</page><cookieJar></cookieJar><agent>Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.111 Safari/537.36</agent></regex><regex><name>video</name><expres>$pyFunction:'[makelist.param2]'.replace(';','&amp;')</expres><page></page></regex>
<info>Futbol videoları ve maçların gollerini izleyin. Süper Lig maç özetleri maçlardan hemen sonra buradan bakabilirsiniz, futbolun kalbi burada atıyor</info><thumbnail>https://www.dropbox.com/s/18ejwn03vdj/macozet1.jpg?dl=1</thumbnail><fanart>https://www.dropbox.com/s/f6ndts69op74g6q/beinwpaper.png?dl=1</fanart></item>
 

undpnd tqucbo

Member
Jan 25, 2021
115
0
16
Can I get help for site & tk?
Code:
https://tv.vera.com.uy/play/2s68l
 

m5Germany

Active member
Mar 20, 2014
281
87
28
So, the Key is the API Token d5q60x0jaOQB3A98A4SSPjww
I tried to write a Post RegEx (below) but does not work. Still learning.
If someone would write the Post request - it will make the Code complete.
The Code should work until the token changes.
LSP3 XML:
<item>
<title>Al_Jazeera RegEx</title>
<fanart></fanart>
<thumbnail></thumbnail>
<link>$doregex[m5]|User-Agent=Mozilla/5.0&amp;Accept=*/*&amp;Origin=https://tv.vera.com.uy&amp;Referer=https://tv.vera.com.uy/</link>
<regex>
<name>m5</name>
<expres>url":"([^"]+)</expres>
<page><![CDATA[https://veratv-be.vera.com.uy/api/setup?token=d5q60x0jaOQB3A98A4SSPjww&public_id=2s68l]]></page>
<accept>application/json</accept>
<agent>Mozilla/5.0</agent>
<referer>https://tv.vera.com.uy/</referer>
</regex>
</item>
 
<item>
<title>RawPost not working</title>
<link>https:$doregex[getAPI]</link>
<regex>
<name>getAPI</name>
<expres>token":"([^"]+)</expres>
<page>https://veratv-be.vera.com.uy/api/sesiones</page>
<accept>application/json</accept>
<agent>Mozilla/5.0</agent>
<referer>https://tv.vera.com.uy/</referer>
<origin>https://tv.vera.com.uy</origin>
<rawpost>tipo:anonima</rawpost>
</regex>
</item>
 
  • Like
Reactions: undpnd tqucbo

gujal

Developer
Dec 26, 2013
564
311
63
Enjoy!

LSP XML:
<item>
<title>Al_Jazeera</title>
<link>$doregex[getUrl]|User-Agent=iPad</link>
<regex>
<name>getUrl</name>
<expres><![CDATA[#$pyFunction
def GetLSProData(page_data,Cookie_Jar,m,url="https://tv.vera.com.uy/play/2s68l"):
  import re, requests
  tokenurl = "https://veratv-be.vera.com.uy/api/sesiones"
  headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:85.0) Gecko/20100101 Firefox/85.0',
             'Origin': 'https://tv.vera.com.uy',
             'Referer': 'https://tv.vera.com.uy/'}
  pdata = {"tipo":"anonima"}
  jd = requests.post(tokenurl, json=pdata, headers=headers).json()
  curl = "https://veratv-be.vera.com.uy/api/setup?token={1}&public_id={0}"
  curl = curl.format(url.split('/')[-1], jd.get('token'))
  data = requests.get(curl, headers=headers).json()
  return data.get('url').get('available')[0].get('playbackUrl').get('url')
]]></expres>
<page></page>
</regex>
</item>
 

gujal

Developer
Dec 26, 2013
564
311
63
And here is the full makelist. It plays mosts channels unless geo-restricted to Uruguay. Enjoy!
LSP XML:
<item>
<title>Vera TV</title>
<link>$doregex[makelist]</link>
<regex>
<name>makelist</name>
<listrepeat><![CDATA[
<title>[COLOR gold][makelist.param1][/COLOR]</title>
<link>$doregex[getUrl]|User-Agent=iPad</link>
<thumbnail>[makelist.param2]</thumbnail>
<info>[makelist.param3]</info>
]]></listrepeat>
<expres><![CDATA[#$pyFunction
def GetLSProData(page_data,Cookie_Jar,m):
  import requests
  tokenurl = "https://veratv-be.vera.com.uy/api/sesiones"
  headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:85.0) Gecko/20100101 Firefox/85.0',
             'Origin': 'https://tv.vera.com.uy',
             'Referer': 'https://tv.vera.com.uy/'}
  pdata = {"tipo":"anonima"}
  jd = requests.post(tokenurl, json=pdata, headers=headers).json()
  token = jd.get('token')
  headers.update({
    "x-service-id": str(jd.get('id_servicio')),
    "x-frontend-id": str(jd.get('id_frontend')),
    "x-system-id": "1",
    "authorization": "Bearer " + jd.get('jwt')
  })
  curl = "https://cds-frontend.vera.com.uy/api-contenidos/listas/68?token={0}"
  curl = curl.format(token)
  jd = requests.get(curl, headers=headers).json()
  data = [(x['nombre'], x['imagen_principal'], x['descripcion'], token, x['public_id']) for x in jd.get('contenidos')]
  return data
]]></expres>
<page></page>
</regex>
<regex>
<name>getUrl</name>
<expres><![CDATA[#$pyFunction
def GetLSProData(page_data,Cookie_Jar,m):
  import requests
  headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:85.0) Gecko/20100101 Firefox/85.0',
             'Origin': 'https://tv.vera.com.uy',
             'Referer': 'https://tv.vera.com.uy/'}
  curl = "https://veratv-be.vera.com.uy/api/setup?token={0}&public_id={1}"
  curl = curl.format('[makelist.param4]', '[makelist.param5]')
  data = requests.get(curl, headers=headers).json()
  return data.get('url').get('available')[0].get('playbackUrl').get('url')
]]></expres>
<page></page>
</regex>
</item>
 
  • Like
Reactions: m5Germany

m5Germany

Active member
Mar 20, 2014
281
87
28
Gujal, Thanks for the Python Block. Still not 100% on this - but- was able to understand
what your Code was/is doing. Took a little different approach to test my theory/thinking.
Used your Python Block to get the "curl" and then used RegEx for final.
Also, did a search for/of "curl.format(url.split('/')[-1]" and getting closer to fully
understanding - but - not there yet. Working on it. Thanks, m5
LSP3 XML:
<item>
<title>Al_Jazeera Redux</title>
  <link>$doregex[m5]|User-Agent=iPad</link>
    <regex>
      <name>m5</name>
      <expres>playbackUrl".."url":"([^"]+)</expres>
      <page>$doregex[getUrl]</page>
      <accept>application/json</accept>
      <agent>Mozilla/5.0</agent>
      <referer>https://tv.vera.com.uy/</referer>
    </regex>
    <regex>
      <name>getUrl</name>
      <expres><![CDATA[#$pyFunction
def GetLSProData(page_data,Cookie_Jar,m,url="https://tv.vera.com.uy/play/2s68l"):
  import re, requests
  tokenurl = "https://veratv-be.vera.com.uy/api/sesiones"
  headers = {'User-Agent': 'Mozilla/5.0',
             'Origin': 'https://tv.vera.com.uy',
             'Referer': 'https://tv.vera.com.uy/'}
  pdata = {"tipo":"anonima"}
  jd = requests.post(tokenurl, json=pdata, headers=headers).json()
  curl = "https://veratv-be.vera.com.uy/api/setup?token={1}&public_id={0}"
  curl = curl.format(url.split('/')[-1], jd.get('token'))
  return curl
]]></expres>
<page />
</regex>
</item>
 

gujal

Developer
Dec 26, 2013
564
311
63
@m5Germany
the split is just getting the last bit which is the stream id. see below
curl.format() basically replaces {0} and {1} with the variables mentioned. see the effect below when applied
Code:
C:\Users\gujal>python

Python 3.8.6 (tags/v3.8.6:db45529, Sep 23 2020, 15:52:53) [MSC v.1927 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> url="https://tv.vera.com.uy/play/2s68l"
>>> url.split('/')[-1]
'2s68l'
>>>
>>> curl = "https://veratv-be.vera.com.uy/api/setup?token={1}&public_id={0}"
>>> token = "blahblahblah"
>>> curl = curl.format(url.split('/')[-1], token)
>>> curl
'https://veratv-be.vera.com.uy/api/setup?token=blahblahblah&public_id=2s68l'
>>>
 
  • Like
Reactions: m5Germany

m5Germany

Active member
Mar 20, 2014
281
87
28
Gujal, I think I've got it? Thanks, m5
Info:
https://     [-3]     /[-2]/[-1]
h tps://tv.vera.com.uy/play/2s68l
url.split('/')[-1]   returns   2s68l
url.split('/')[-2]   returns   play
url.split('/')[-3]   returns   tv.vera.com.uy
https://   returns   everything after //
url.split('//')[-1]  returns   tv.vera.com.uy/play/2s68l
 

Doom

Member
Sep 16, 2019
317
21
18
Stopped working can i get a fix and thanks!
LSP XML:
<item>
<title>Azteca Cinema</title>
<thumbnail>https://www.televisiongratishd.com/img/azcinema.png</thumbnail>
<fanart>https://www.metropoliabierta.com/uploads/s1/23/02/01/captura-de-pantalla-2018-05-07-a-las-15-47-45.png</fanart>
<link>$doregex[getUrl]|user-agent=iPad&amp;Referer=https://live.televisiongratishd.com/&amp;verify=False</link>
<regex>
<name>getUrl</name>
<expres><![CDATA[#$pyFunction
def GetLSProData(page_data,Cookie_Jar,m):
  import re, requests, base64, jsunpack
  link = re.findall('<iframe\s*src=[\'"]([^#]+)', page_data)[0]
  page_data = requests.get(link,headers={'user-agent':'Mozilla/5.0','referer':'https://www.televisiongratishd.com/live/azcinema1.php?id=54_','accept':'*/*'}, verify=False).text
  pdata = re.search('(eval\(function\(p,a,c,k,e.+?)</script', page_data, re.DOTALL)
  if pdata:
    page_data += jsunpack.unpack(pdata.group(1))
  burl = re.findall('MARIOCSCryptOld\("([^"]+)', page_data)[0]
  while 'm3u8' not in burl:
    burl = base64.b64decode(burl).decode('utf8')
  return burl
]]></expres>
<page>https://www.televisiongratishd.com/live/azcinema1.php?id=54_</page>
<referer>https://www.televisiongratishd.com/live/azcinema.php</referer>
</regex>
</item>
 

gujal

Developer
Dec 26, 2013
564
311
63
Enjoy!

LSP XML:
<item>
<title>Azteca Cinema</title>
<thumbnail>https://www.televisiongratishd.com/img/azcinema.png</thumbnail>
<fanart>https://www.metropoliabierta.com/uploads/s1/23/02/01/captura-de-pantalla-2018-05-07-a-las-15-47-45.png</fanart>
<link>$doregex[getUrl]|user-agent=iPad&amp;Referer=$doregex[getIframe]&amp;verifypeer=false</link>
<regex>
<name>getUrl</name>
<expres><![CDATA[#$pyFunction
def GetLSProData(page_data,Cookie_Jar,m):
  import re, base64, jsunpack
  pdata = re.search('(eval\(function\(p,a,c,k,e.+?)</script', page_data, re.DOTALL)
  if pdata:
    page_data += jsunpack.unpack(pdata.group(1))
  burl = re.findall('MARIOCSCryptOld\("([^"]+)', page_data)[0]
  while 'm3u8' not in burl:
    burl = base64.b64decode(burl).decode('utf8')
  return burl
]]></expres>
<page>$doregex[getIframe]</page>
<referer>https://www.televisiongratishd.com/live/azcinema.php</referer>
</regex>
<regex>
<name>getIframe</name>
<expres>iframe\s*src=[\'"]([^#"']+)</expres>
<page>https://www.televisiongratishd.com/live/azcinema1.php?id=54_</page>
<referer>https://www.televisiongratishd.com/live/azcinema.php</referer>
</regex>
</item>
 

gujal

Developer
Dec 26, 2013
564
311
63
I presume you are using it on Kodi Leia, and yes it wont work there as the cert issuer for telelatino website is not trusted by Leia
The certifi module on Leia version is 2019.9.11 which is quite old
It works fine on Matrix as certifi version is 2020.12.05
Cant do much unless certifi module is updated for Leia (I dont think that will happen)
 

gujal

Developer
Dec 26, 2013
564
311
63
Why run buggy 19 Alpha 3? if you are on 19 you might as well be running the 19.1 released version and on that the above code works fine
 

Catmantx

Member
Nov 2, 2020
65
3
8
Hello, I think is simple but can't get it o work with old scripts.
Can you please make this on?

Thank you in advance!!!
Code:
hxxps://filme24.club/embed/eurosport-1.html
 

gujal

Developer
Dec 26, 2013
564
311
63
Enjoy!

LSP XML:
<item>
<title>Eurosport 1</title>
<link>$doregex[getUrl]|User-Agent=iPad&amp;Referer=https://filme24.club/</link>
<thumbnail>https://discovery-assets-production.s3.eu-west-1.amazonaws.com/app/uploads/2019/04/03130804/logo-eurosport-1-hd.jpg</thumbnail>
<regex>
<name>getUrl</name>
<expres>source:\s*'([^']+)</expres>
<page>https://filme24.club/embed/eurosport-1.html</page>
<referer>https://filme24.club/</referer>
<cookiejar>$doregex[getCookie]</cookiejar>
</regex>
<regex>
<name>getCookie</name>
<expres></expres>
<page>https://filme24.club/tv-live/?url=eurosport-1</page>
<referer>https://filme24.club/</referer>
<cookiejar></cookiejar>
</regex>
</item>
 
  • Like
Reactions: Catmantx

nkt9911

New member
May 22, 2021
21
0
1
USA
Hi.
Can anyone help me with the site?
Code:
https://ztv.uz/online-tv/sd/121-kinoteatr.html
 

m5Germany

Active member
Mar 20, 2014
281
87
28
LSP3 XML:
<item>
<title>Kinoteatr ztv.uz</title>
<fanart>https://ohcbrands.com/wp-content/uploads/2018/05/black-background.jpg</fanart>
<thumbnail>https://ztv.uz/templates/Default/images/logo.png</thumbnail>
<link>$doregex[m5]|User-Agent=Mozilla/5.0&amp;Accept=*/*&amp;Origin=https://ztv.uz&amp;Referer=https://ztv.uz/</link>
<regex>
<name>m5</name>
<expres>embedUrl":\s*"([^"]+)</expres>
<page>https://ztv.uz/online-tv/sd/121-kinoteatr.html</page>
<connection>keep-alive</connection>
<agent>Mozilla/5.0</agent>
<accept>text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9</accept>
<referer>https://ztv.uz/</referer>
<cookiejar>$doregex[getCookie]</cookiejar>
</regex>
<regex>
<name>getCookie</name>
<expres></expres>
<page>https://ztv.uz/online-tv/sd/121-kinoteatr.html1</page>
<referer>https://ztv.uz/</referer>
<cookiejar></cookiejar>
</regex>
</item>
 
  • Like
Reactions: nkt9911
Thread starter Similar threads Forum Replies Date
G Scraper Development 0
J Scraper Development 1
O Addon Requests 5