LSP Regex & Makelist Discussion.

sagheerahmed94

New member
Apr 13, 2013
6
1
3
NEW YORK
@gujal,
I came to this forum after a long time and i was very happy to see you and many others who were in the old forum. you may remember that you did make a lot of regex in the old forum and you will be wounder that i still have some regex you did make for me, i kept those regex just for old rememberence. you guys are rock like pmolson, shani and streamwatcher i still remember and miss you guys.
 

m5Germany

Active member
Mar 20, 2014
329
105
43
sagheerahmed94
Found this from day's gone by. @pmolson helped me with this "test" Code.
LSP XML:
<item>
<title>(02/10/2019-m5) [USER=3264]@pmolson[/USER] | RegEx WORKING w/calls/get/source | streamgaroo.com</title>
<link>$doregex[getFinalM3u8]|Referer=http://www.streamgaroo.com/player/GrindPlayer.swf&amp;User-Agent=Mozilla/5.0</link>
<regex>
<name>getFinalM3u8</name>
<expres>hls.{2}["'](http[^'"]*)["']</expres>
<page>$doregex[getCleanM3u8Page]</page>
</regex>
<regex>
<name>getCleanM3u8Page</name>
<expres>$pyFunction:'$doregex[getM3u8Page]'.replace('\/','/')</expres>
<page></page>
</regex>
<regex>
<name>getM3u8Page</name>
<expres>["'](http[^'"]*)["']</expres>
<page>http://www.streamgaroo.com$doregex[get_source]</page>
<referer>http://www.streamgaroo.com/</referer>
<x-req>XMLHttpRequest</x-req>
<rawpost>h=$doregex[get-rawpost]</rawpost>
<cookieJar></cookieJar>
<agent>Mozilla/5.0</agent>
</regex>
<regex>
<name>get_source</name>
 
<expres>root\+"(.*?source)"</expres>
<info>root\+"([^"]*source)</info>
 
 
<page>http://www.streamgaroo.com/js/stream.min.js</page>
<referer>http://www.streamgaroo.com/</referer>
<cookieJar></cookieJar>
<agent>Mozilla/5.0</agent>
</regex>
<regex>
<name>get-rawpost</name>
 
<expres>="true" data-h="([^"]*==)"</expres>
<info>data-sh="([^"]*)"</info>
 
<page>https://www.streamgaroo.com/television/united-states/cnn-live-stream</page>
<referer>http://www.streamgaroo.com/</referer>
<agent>Mozilla/5.0</agent>
<cookieJar></cookieJar>
</regex>
</item>
This is when I started trying to learn. Still "Learning"
Thanks,
m5
 
  • Like
Reactions: sagheerahmed94

m5Germany

Active member
Mar 20, 2014
329
105
43
And this:
Info:
<item>
<title>ORIGINAL | streamlive.to if / else | @Shani_08</title>
<link>$doregex[select_Url]</link>
<regex><name>select_Url</name>
<expres>$pyFunction: '$doregex[get_HD]' if not '$doregex[get_HD]'=='' else '$doregex[get_SD]'</expres><page></page></regex><regex>
<name>get_HD</name>
<expres>$pyFunction:vipracing.decrypt_vipracing('http://www.streamlive.to/view/69310',False,False)</expres><page></page></regex>
<regex><name>get_SD</name>
<expres>$pyFunction:vipracing.decrypt_vipracing('http://www.streamlive.to/view/71562',False,False)</expres><page></page></regex>
</item>
Long time ago.
 
  • Like
Reactions: sagheerahmed94

RRrorony

New member
Jun 17, 2019
25
6
3
Here your code
LSP XML:
<item>
<title>Arena SPORT1</title>
<link>$doregex[getUrl]</link>
<regex>
<name>getUrl</name>
<expres><![CDATA[#$pyFunction
def GetLSProData(page_data,Cookie_Jar,m):
  import requests,re

  url = "http://wizhdsports.net/wiz21.php"
  headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36'}
  ifr = re.findall(r'iframe src="(https[^"]+)',requests.get(url, headers=headers).text)[0]
  headers.update({'referer': 'http://wizhdsports.net/'})
  hls = re.findall(r"source:'([^']+)",requests.get(ifr, headers=headers).text)[0]
  return hls+'|Origin=https://www.wmsxx.com&Referer=https://www.wmsxx.com&User-Agent=Mozilla/5.0'
]]></expres>
<page></page>
</regex>
</item>
 
  • Like
Reactions: undpnd tqucbo

undpnd tqucbo

Member
Jan 25, 2021
160
2
18
Need help this site.Tk
Code:
https://www.ustream.to/live?id=ziggo-sport-voetbal
 

m5Germany

Active member
Mar 20, 2014
329
105
43
Greetings, I tried to Code this w/ no success. Not sure why it doesn't work. Strange ERROR:error probing input format (see below).
Where did: "https://img.olcdn.net/hls_error.php?url=" come from?
Info:
<item>
<title>Test - www.ustream.to</title>
<thumbnail></thumbnail>
<fanart></fanart>
<link>https://hls.ustream.to/Ziggo-Sport-Voetbal.m3u8?token=$doregex[2]|User-Agent=Mozilla/5.0</link>
<regex>
<name>2</name>
<expres><![CDATA[jdtk="([^"]+)]]></expres>
<page>$doregex[1]</page>
<referer>https://beef1999.blogspot.com/</referer>
</regex>
<regex>
<name>1</name>
<expres>$pyFunction:get_unpacked(page_data,'(eval.*?\}\)\)).*\n.*\n.*\nvar cdn_loaded = 1;')</expres>
<page>https://www.ustream.to/stream_original.php?id=ziggo-sport-voetbal</page>
<referer>https://beef1999.blogspot.com/</referer>
</regex>
</item>

NOTICE: VideoPlayer::OpenFile: plugin://plugin.video.live.streamspro/?url=https%3A%2F%2Fhls.ustream.to%2FZiggo-Sport-Voetbal.m3u8%3Ftoken%3D%24doregex%5B2%5D%7CUser-Agent%3DMozilla%2F5.0&mode=17&regexs=%7Bu%271%27%3A%20%7B%27expres%27%3A%20u%22%24pyFunction%3Aget_unpacked%28page_data%2C%27%28eval.%2A%3F%5C%5C%7D%5C%5C%29%5C%5C%29%29.%2A%5C%5Cn.%2A%5C%5Cn.%2A%5C%5Cnvar%20cdn_loaded%20%3D%201%3B%27%29%22%2C%20%27referer%27%3A%20u%27https%3A//beef1999.blogspot.com/%27%2C%20%27name%27%3A%20u%271%27%2C%20%27page%27%3A%20u%27https%3A//www.ustream.to/stream_original.php%3Fid%3Dziggo-sport-voetbal%27%7D%2C%20u%272%27%3A%20%7B%27expres%27%3A%20u%27jdtk%3D%22%28%5B%5E%22%5D%2B%29%27%2C%20%27referer%27%3A%20u%27https%3A//beef1999.blogspot.com/%27%2C%20%27name%27%3A%20u%272%27%2C%20%27page%27%3A%20u%27%24doregex%5B1%5D%27%7D%7D
NOTICE: Creating InputStream
NOTICE: Creating Demuxer
ERROR: CDVDDemuxFFmpeg::Open - error probing input format, https://img.olcdn.net/hls_error.php?url=https://hls.ustream.to/Ziggo-Sport-Voetbal.m3u8?token=027-bf4-642-7b1-cd1-d2f-9a1-8a1-41d-38d-2ef-b71-c0e-7a4-c7e-384-84a-9a1-6ea-404-5d7-4&referer=&origin=
ERROR: CVideoPlayer::OpenDemuxStream - Error creating demuxer
NOTICE: CVideoPlayer::OnExit()
NOTICE: CVideoPlayer::CloseFile()
NOTICE: VideoPlayer: waiting for threads to exit
NOTICE: VideoPlayer: finished waiting
As Always, Thanks.
m5
 

streamwatcher2013

Well-known member
Oct 24, 2013
986
446
63
USA
Try this:

LSP XML:
<item>
<title>https://www.ustream.to/live?id=ziggo-sport-voetbal</title>
<link>$doregex[getUrl]</link>
<regex>
<name>getUrl</name>
<expres><![CDATA[#$pyFunction
import re,jsunpack
def GetLSProData(page_data,Cookie_Jar,m):
  list=jsunpack.unpack(re.findall('(eval.*(?:m3u8|atob).*)',page_data)[0])
  netloc,path,html=re.findall('file_name.+?"([^"]+).+?jdtk.+?"([^"]+).+?host_tmg.+?"([^"]+)',list)[0]
  return 'https://%s/%s?token=%s|user-agent=ipad&origin=https://ustreamix.to&referer=https://www.ustream.to/stream_original.php?id=ziggo-sport-voetbal'%(html,netloc,path)
]]></expres>
<page>https://www.ustream.to/stream_original.php?id=ziggo-sport-voetbal</page>
<referer>https://www.ustream.to/live?id=ziggo-sport-voetbal</referer>
</regex>
</item>
 

m5Germany

Active member
Mar 20, 2014
329
105
43
Not sure why - but- when I use the Code it caused Kodi to crash - multiple times?
Info:
NOTICE: VideoPlayer::OpenFile: plugin://plugin.video.live.streamspro/?url=%24doregex%5BgetUrl%5D&mode=17&regexs=%7Bu%27getUrl%27%3A%20%7B%27expres%27%3A%20u%27%23%24pyFunction%5Cnimport%20re%2Cjsunpack%5Cndef%20GetLSProData%28page_data%2CCookie_Jar%2Cm%29%3A%5Cn%20%20list%3Djsunpack.unpack%28re.findall%28%5C%27%28eval.%2A%28%3F%3Am3u8%7Catob%29.%2A%29%5C%27%2Cpage_data%29%5B0%5D%29%5Cn%20%20netloc%2Cpath%2Chtml%3Dre.findall%28%5C%27file_name.%2B%3F%22%28%5B%5E%22%5D%2B%29.%2B%3Fjdtk.%2B%3F%22%28%5B%5E%22%5D%2B%29.%2B%3Fhost_tmg.%2B%3F%22%28%5B%5E%22%5D%2B%29%5C%27%2Clist%29%5B0%5D%5Cn%20%20return%20%5C%27https%3A//%25s/%25s%3Ftoken%3D%25s%7Cuser-agent%3Dipad%26origin%3Dhttps%3A//ustreamix.to%26referer%3Dhttps%3A//www.ustream.to/stream_original.php%3Fid%3Dziggo-sport-voetbal%5C%27%25%28html%2Cnetloc%2Cpath%29%5Cn%27%2C%20%27referer%27%3A%20u%27https%3A//www.ustream.to/live%3Fid%3Dziggo-sport-voetbal%27%2C%20%27name%27%3A%20u%27getUrl%27%2C%20%27page%27%3A%20u%27https%3A//www.ustream.to/stream_original.php%3Fid%3Dziggo-sport-voetbal%27%7D%7D
NOTICE: Creating InputStream
NOTICE: Creating Demuxer
NOTICE: Opening stream: 0 source: 256
NOTICE: Creating video codec with codec id: 61
NOTICE: CDVDVideoCodecFFmpeg::Open() Using codec: PNG (Portable Network Graphics) image
NOTICE: Creating video thread
NOTICE: running thread: video_thread
NOTICE: CVideoPlayer::OnExit()
NOTICE: VideoPlayer: eof, waiting for queues to empty
NOTICE: Closing stream player 2
NOTICE: waiting for video thread to exit
ERROR: Got MSGQ_ABORT or MSGO_IS_ERROR return true
NOTICE: thread end: video_thread
NOTICE: deleting video codec
No clue - just wanted to let you know. Not sure if others experience the same crash.
m5
 

m5Germany

Active member
Mar 20, 2014
329
105
43
Although I am not that familiar w/ Firefox browser - the problem w/ the site is: While using Firefox Browser dev tools network results.
Info:
https://www.revenuecpmnetwork.com/d3/ca/55/d3ca554b4225392bdce7d2a327f00def.js

Status499
Request has been forbidden by antivirus
VersionHTTP/1.1
Transferred11.98 KB (11.75 KB size)
Referrer Policyno-referrer-when-downgrade
Normally use view-source: or Dev Tools(Chrome) and Fiddler 5.0.2+ It appears that the site will not grant access unless you set browser to accept ads and turn off your Security.
Well, for me - just trying to Code the stream and having no Security is not worth it.
Personally, I am done with this challenge and ready for the next.
Please do not feel like you need to pursue this stream Code. It's 1 stream. Not worth it.

I do want to Thank You for all the support you have shown this forum/thread over the years.
m5
 
  • Like
Reactions: JonnyB

tpoupi

Member
Jan 2, 2016
38
0
6
Please need regex. Thank so much
hxxps://www.qub.ca/tvaplus/tva/en-direct
 

Doux ami

New member
Oct 25, 2015
5
1
3
Canada
Salut tpoupi,
Tu n'as pas besoin de regex étant donné que la chaîne fonctionne sans token donc c'est un lien statique.

Code:
<item>
<title>TVA</title>
<link>https://tvalive.akamaized.net/hls/live/2012413/tva01/high_1080p/high_1080p.m3u8|User-Agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.111 Safari/537.36</link>
</item>
Salut au passage à Rorony, bien content que tu sois revenu parmi nous!
 

Catmantx

Member
Nov 2, 2020
67
3
8
Something changed yesterday on YT. The regex does not pull the playlist anymore, Please take a look when you have a chance.



Code:
<item>
<title>[COLOR yellow][B]Telejurnalul Orei 20[/B][/COLOR]</title>
<thumbnail>https://i.ibb.co/mX8MCb9/TVRjurnal.png</thumbnail>
<link>$doregex[makelist]</link>
<regex>
<name>makelist</name>
<listrepeat><![CDATA[
<title>[COLOR gold][makelist.param1] [COLOR cyan]([makelist.param2])[/COLOR]</title>
<info>[makelist.param3]</info>
<urlsolve>https://www.youtube.com/watch?v=[makelist.param4]</urlsolve>
<thumbnail>[makelist.param5]</thumbnail>
]]></listrepeat>
<expres><![CDATA[#$pyFunction
def GetLSProData(page_data, Cookie_Jar, m, list='UUlB-stfNogHt7kPR-p_lppA'):
  import requests
  headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0'}
  apiurl = 'https://www.youtube.com/list_ajax?style=json&action_get_list=1&list={0}&index={1}'
  index = 1
  events = []
  while True:
    page_data = requests.get(apiurl.format(list, index), headers=headers).json()
    items = page_data.get('video')
    index += 200
    for item in items:
      title = item.get('title')
      vid = item.get('encrypted_id')
      info = item.get('description')
      length = item.get('duration')
      thumb = item.get('thumbnail').replace('default', 'hqdefault')
      event = (title, length, info, vid, thumb)
      if event not in events:
        events.append(event)
    if len(events) % 200 != 0:
      return events
]]></expres>
<page></page>
</regex>
</item>

and the same with this
Code:
<item>
<title>[COLOR yellow]TUDN[/COLOR][COLOR red][B]  [YT][/B][/COLOR]   [US only]</title>
<link>$doregex[makelist]</link>
<regex>
<name>makelist</name>
<listrepeat><![CDATA[
  <title>[COLOR gold][makelist.param3][/COLOR]</title>
  <link>$doregex[makelist2]</link>
  <thumbnail>[makelist.param2]</thumbnail>
]]></listrepeat>
<expres><![CDATA[#$pyFunction
import re,urllib
def GetLSProData(page_data, Cookie_Jar, m):#streamwatcher2013
  source=urllib.urlopen('https://www.youtube.com/c/TUDNUSA/playlists').read()
  return re.findall('"gridPlaylistRenderer":{"playlistId":"([^"]+).+?"thumbnails":.{"url":"([^\?]+).+?"title":{"runs":.{"text":"([^"]+)',source)
]]></expres>
<page></page>
</regex>
<regex>
<name>makelist2</name>
<listrepeat><![CDATA[
  <title>[COLOR gold][makelist2.param1] [COLOR cyan]([makelist2.param2])[/COLOR]</title>
  <info>[makelist2.param3]</info>
  <urlsolve>https://www.youtube.com/watch?v=[makelist2.param4]</urlsolve>
  <thumbnail>https://i.ytimg.com/vi/[makelist2.param4]/hqdefault.jpg</thumbnail>
]]></listrepeat>
<expres><![CDATA[#$pyFunction
import re,requests
def GetLSProData(page_data, Cookie_Jar, m):#streamwatcher2013
  count=1;pn=1;jdata=[]
  while pn <= int(count):
    source=requests.get('https://www.youtube.com/list_ajax?style=json&action_get_list=1&list=[makelist.param1]&index='+str(pn),headers={'user-agent':'Mozilla/5.0'}).json()
    jdata +=source.get('video');pn +=1
    channels=[]
    for j in jdata:
      title=j.get('title')
      vid=j.get('encrypted_id')
      inf=j.get('description')
      length=j.get('duration')
      channels.append((title,length,inf,vid))
  return channels
]]></expres>
<page></page>
</regex>
<thumbnail>https://yt3.ggpht.com/a/AATXAJy0sIkIpXR0saqRlzXwrseDY9FXGqNDj8-5i365VGQ=s88-c-k-c0x00ffffff-no-rj</thumbnail>
</item>
one:
 

Catmantx

Member
Nov 2, 2020
67
3
8
sorry, this is the problem that doesn't work
Code:
https://www.youtube.com/list_ajax?style=json&action_get_list=1&list=UUlB-stfNogHt7kPR-p_lppA&index=1
 

arslion84

New member
Dec 31, 2015
8
0
1
Hi
XML:
<item>
<title>Cukar[/COLOR]</title>
<link>$doregex[makelist]</link>
<regex>
<name>makelist</name>
<listrepeat><![CDATA[
<title>[COLOR gold][makelist.param1] [COLOR cyan]([makelist.param2])[/COLOR]</title>
<info>[makelist.param3]</info>
<urlsolve>https://www.youtube.com/watch?v=[makelist.param4]</urlsolve>
<thumbnail>[makelist.param5]</thumbnail>
]]></listrepeat>
<expres><![CDATA[#$pyFunction
def GetLSProData(page_data,Cookie_Jar,m):
  import json
  items = json.loads(page_data).get('video')
  events = []
  for item in items:
    title = item.get('title')
    vid = item.get('encrypted_id')
    info = item.get('description')
    length = item.get('duration')
    thumb = item.get('thumbnail').replace('default', 'hqdefault')
    events.append((title, length, info, vid, thumb))
  return events
]]></expres>
            <page>https://www.youtube.com/list_ajax?style=json&amp;action_get_list=1&amp;list=PLytu9wIdhRyQiSyS0p6PSsxYKKhk__d3I</page>
             <cookieJar></cookieJar>
</regex>
<thumbnail>http://aslininsureti.com/wp-content/uploads/2017/10/yama%C3%A7-ko%C3%A7oval%C4%B1-aras-bulut-iynemli-%C3%A7ukur-1c.jpg</thumbnail><fanart>http://www.pixelstalk.net/wp-content/uploads/2016/07/Pink-And-Black-HD-Background.jpg</fanart></item>
i have also Problem with the Youtube regex, please for fix this..
Thanks in Advance
 

streamwatcher2013

Well-known member
Oct 24, 2013
986
446
63
USA
@Catmantx and @arslion84

You need to get the latest of plugin.youtube alpha version same as the latest version of Resolveurl or Urlresolve (also for who is having problem playing youtube in Kodi). This is just temporary workout for now with up to 100 channels until @gujal come up with the solution to get more than 100 channels. The POST is complicated.

LSP XML:
<item>
<title>Cukar Bölüm</title>
<link>$doregex[makelist]</link>
<regex>
<name>makelist</name>
<listrepeat><![CDATA[
  <title>[COLOR gold]&#9818; [makelist.param1] [COLOR dodgerblue]([makelist.param2])[/COLOR]</title>
  <info>Thời lượng [makelist.param2]</info>
  <urlsolve>https://www.youtube.com/watch?v=[makelist.param3]</urlsolve>
  <thumbnail>https://i.ytimg.com/vi/[makelist.param3]/hqdefault.jpg</thumbnail><fanart>https://yt3.ggpht.com/Gp1vjeWqBOsL3s4He_i34LZtHwcULTTjDdrEZmh0lDJOe5HoGRMgorPCJwKsnqdKTdav_Hyb2g=w2120-fcrop64=1,00005a57ffffa5a8-k-c0xffffffff-no-nd-rj</fanart>
]]></listrepeat>
<expres><![CDATA[#$pyFunction
import re,urllib
def GetLSProData(page_data,Cookie_Jar,m):
  page_data=urllib.urlopen('https://www.youtube.com/playlist?list=PLytu9wIdhRyQiSyS0p6PSsxYKKhk__d3I').read()
  jdata=re.findall('"sectionListRenderer":{"contents"(?s)(.*?)id="container"',page_data)[0].replace('\u0026','&').replace('\\','')
  YTid=re.findall('"playlistVideoRenderer":{"videoId":"([^"]+)',jdata)
  title=re.findall('"playlistVideoRenderer":{"videoId":[\w\W]+?"accessibilityData":{"label":"(.+?)\s*by',jdata)
  duration=re.findall('"playlistVideoRenderer":{"videoId":[\w\W]+?},"simpleText":"([^"]+)',jdata)
  return zip(title,duration,YTid)
]]></expres>
<page></page>
</regex>
</item>

<item>
<title>TVR1</title>
<link>$doregex[makelist]</link>
<regex>
<name>makelist</name>
<listrepeat><![CDATA[
  <title>[COLOR gold]&#9818; [makelist.param1] [COLOR dodgerblue]([makelist.param2])[/COLOR]</title>
  <info>Thời lượng [makelist.param2]</info>
  <urlsolve>https://www.youtube.com/watch?v=[makelist.param3]</urlsolve>
  <thumbnail>https://i.ytimg.com/vi/[makelist.param3]/hqdefault.jpg</thumbnail><fanart>https://yt3.ggpht.com/Gp1vjeWqBOsL3s4He_i34LZtHwcULTTjDdrEZmh0lDJOe5HoGRMgorPCJwKsnqdKTdav_Hyb2g=w2120-fcrop64=1,00005a57ffffa5a8-k-c0xffffffff-no-nd-rj</fanart>
]]></listrepeat>
<expres><![CDATA[#$pyFunction
import re,urllib
def GetLSProData(page_data,Cookie_Jar,m):
  page_data=urllib.urlopen('https://www.youtube.com/playlist?list=UUlB-stfNogHt7kPR-p_lppA').read()
  jdata=re.findall('"sectionListRenderer":{"contents"(?s)(.*?)id="container"',page_data)[0].replace('\u0026','&').replace('\\','')
  YTid=re.findall('"playlistVideoRenderer":{"videoId":"([^"]+)',jdata)
  title=re.findall('"playlistVideoRenderer":{"videoId":[\w\W]+?"accessibilityData":{"label":"(.+?)"}',jdata)
  duration=re.findall('"playlistVideoRenderer":{"videoId":[\w\W]+?},"simpleText":"([^"]+)',jdata)
  return zip(title,duration,YTid)
]]></expres>
<page></page>
</regex>
</item>


 

m5Germany

Active member
Mar 20, 2014
329
105
43
Greetings, Not sure the following is possible - but - where else would I ask it?
Is there a way to Code: That then when: there is an instance of RegEx'd value and that value is captured multiple times - to display/play just the 1st instance? It's semi-annoying when multiple repeat same Code shows in list - Confluence Skin used.
Looking to capture 1st instance of same Code once and move on to next. Capture/Skip should not be Regex’d as they
are only to show what is to be captured.
Info:
value http://333.33/22/11a capture
value http://333.33/22/11a skip
value http://333.33/22/11b capture
value http://333.33/22/11b skip  
value http://333.33/22/11c capture
value http://333.33/22/11c skip
I would like to capture 1st of a b c and not display/play 2nd a b c with same Code.
As Always, Thanks in advance.
m5
 

streamwatcher2013

Well-known member
Oct 24, 2013
986
446
63
USA
Try this:

LSP XML:
<item>
<title>Prevent duplicate</title>
<link>$doregex[makelist]</link>
<regex>
<name>makelist</name>
<listrepeat><![CDATA[
  <title>[makelist.param1]</title>
  <link>$doregex[makelist2]</link>
]]></listrepeat>
<expres><![CDATA[#$pyFunction
import re
def GetLSProData(page_data,Cookie_Jar,m):
  source="""{"value":"http://333.33/22/11a"},
             {"value":"http://333.33/22/11a"},
             {"value":"http://333.33/22/11b"},
             {"value":"http://333.33/22/11b"},
             {"value":"http://333.33/22/11c"},
             {"value":"http://333.33/22/11c"}"""
  links=re.findall('"value":"([^"]+)',source)
  return set(links)
]]></expres>
<page></page>
</regex>
</item>
 
  • Like
Reactions: m5Germany

m5Germany

Active member
Mar 20, 2014
329
105
43
Special Thanks to: @RRrorony, @streamwatcher2013 for helping me get this far.

Greetings, Still trying to perfect this Code. Is there a way to incorporate "Prevent duplicate" from above into the Code below?
Still filtering the servers I want to include\restrict. Would like to keep filtering as written - meaning l1 and l2.
ss":"[1] - basically means only play if - liveliness":"100". Eventually, I would like to add streams that are 98-100 as these normally play.
LSP XML:
<item>
<title>[COLOR=FF00FFFF]Main Menu [COLOR=red]•[/COLOR][COLOR=blue]•[/COLOR][COLOR=FF00CCFF] iptvcat.com[/COLOR][/COLOR]</title>
<thumbnail>https://iptvcat.com/res/images/logo_iptvcat.png</thumbnail>
<fanart>https://hlscat.com/res/images/logo_hlscat.png</fanart>
<link>$doregex[makelist]</link>
<regex>
<name>makelist</name>
<listrepeat><![CDATA[
  <title>[COLOR=FF00FFFF][makelist.param1][COLOR=red]•[/COLOR][COLOR=blue]•[/COLOR][COLOR=FF00CCFF] iptvcat.com[/COLOR][/COLOR]</title>
  <thumbnail>https://iptvcat.com/res/images/logo_iptvcat.png</thumbnail>
  <fanart>https://hlscat.com/res/images/logo_hlscat.png</fanart>
  <link>[makelist.param2]|User-Agent=VLC&amp;Referer=http://iptvcat.com/</link>
]]></listrepeat>
<expres><![CDATA[#$pyFunction
import re
import requests
def GetLSProData(page_data,Cookie_Jar,m):
  l1 = re.findall('"channel":"([^\*"]+)","link":"(h.{6}[1acensuv][^18bcdfhimotvx][^ikmr].*?).checkedby.*?ss":"[1]',page_data)
  l2 = re.findall('"channel":"([^\*"]+)","link":"(https:\/\/[^bcdglmpsvw].*?m3u8.*?).checkedby.*?ss":"[1]',page_data)
  l3 = l1+l2
  items = [(x.replace('ISN, Its ','').replace('}','').replace('[','').replace(']','').replace('Baton Rouge LA (WBRZ-HD), WBRZ-TV (Baton Rouge)','WBRZ').replace('|','').replace('ISN','').replace(',','').replace(' (English)','').replace('Pequeradio,','').replace(' - Hastings-On-Hudson - NY','').replace(' (CSULB)','').replace(' (Opt-2)','').replace(' (Opt-3)','').replace('-',' ').replace(' USA','').replace(' US','').replace(' SD','').replace(' (Geo)','').replace('HD (ENG)','').replace('ENG HD','').replace(' ENG','').replace(' HD','').replace(' (ENG)','').replace('\u0026','&'), y.replace('\u0026','&').replace('/u002','')) for x, y in l3]
  return sorted(items,key=lambda t: (set(t[0])))
]]></expres>
<page>https://raw.githubusercontent.com/eliashussary/iptvcat-scraper/master/data/countries/united%20states%20of%20america.json</page>
</regex>
</item>
As Always, Thanks in advance.
m5
 

m5Germany

Active member
Mar 20, 2014
329
105
43
Damn it, Please replace below as I sent the wrong values for l1, l2.
LSP XML:
l1 = re.findall('"channel":"([^\*"]+)","link":"(h.{6}[1acensuv][^18bcdfhimotvx][^mr].*?).checkedby.*?ss":"[1]',page_data)
l2 = re.findall('"channel":"([^\*"]+)","link":"(https:\/\/[^bcdglmpsw].*?m3u8.*?).checkedby.*?ss":"[1]',page_data)
m5
 

Catmantx

Member
Nov 2, 2020
67
3
8
@streamwatcher2013 Thank you very much for your reply [WORKS GREAT]]
if you don't mind, is it possible to shorten the "title"?
 
  • Like
Reactions: arslion84

Dasher

New member
Feb 8, 2021
1
0
1
Hi, I am new to this forum and I have been using LSP for years now I am working on LSP for python3 but I am having issues with lxml parser import
the error is getting "ImportError: DLL load failed: %1 is not a valid Win32 application." this error is because of the file etree with cpython file with dot H extension in it. it cant be imported I don't know how to solve this issue it works fine in my computer but i cant get it to work on xbmc, I want to parse the xml file with element tree addon i have tried but it work parse the "<link>" tag the right way the close is always missing, i dont know if it just my issue or it has to do with element tree but, I cant find a way to parse XML so I have made the add-on base on JSON, this add-on works fine.
 

Catmantx

Member
Nov 2, 2020
67
3
8
Sorry for the abuse but I have one more, please @streamwatcher2013

This one is from this forum too, but is for the whole channel.

Whenever you have some time, please.

Thank you!

XML:
<item>
<title>[COLOR yellow]Sky News[/COLOR][COLOR red][B]  [YT][/B][/COLOR]   [Latest]</title>
<thumbnail>https://yt3.ggpht.com/a/AATXAJzHMjrxBG0TA7AU1HoATeM9DZkmXtb-bmSTidOPL3Q=s100-c-k-c0xffffffff-no-rj-mo</thumbnail>
<fanart>https://yt3.ggpht.com/S4XHNlwIVk7lqA3TZ5GT7WAwDwJtkxkLEA4i5wMjOkaNs3frg6zOfdf4fYtlq1LId67OHMZJvw=w2120-fcrop64=1,00005a57ffffa5a8-k-c0xffffffff-no-nd-rj</fanart>
<link>$doregex[makelist]</link>
<regex>
<name>makelist</name>
<listrepeat><![CDATA[
<title>[COLOR gold][makelist.param1] [COLOR cyan]([makelist.param2])[/COLOR]</title>
<info>[makelist.param3]</info>
<utube>[makelist.param4]</utube>
<thumbnail>[makelist.param5]</thumbnail>
]]></listrepeat>
<expres><![CDATA[#$pyFunction
def GetLSProData(page_data, Cookie_Jar, m, list='UUoMdktPbSTixAyNGwb-UYkQ'):
  import requests
  headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10; Win64; x64; rv:68.0) Gecko/20100101 Firefox/68.0'}
  apiurl = 'https://www.youtube.com/list_ajax?style=json&action_get_list=1&list={0}&index={1}'
  index = 1
  events = []
  while True:
    page_data = requests.get(apiurl.format(list, index), headers=headers).json()
    items = page_data.get('video')
    index += 200
    for item in items:
      title = item.get('title')
      vid = item.get('encrypted_id')
      info = item.get('description')
      length = item.get('duration')
      thumb = item.get('thumbnail').replace('default', 'hqdefault')
      event = (title, length, info, vid, thumb)
      if event not in events:
        events.append(event)
    if len(events) % 200 != 0:
      return events
]]></expres>
<page></page>
</regex>
</item>
 

Catmantx

Member
Nov 2, 2020
67
3
8
Sorry, @streamwatcher2013 I posted the wrong CODE. [Thisis the correct one for YT channel]
Please see below:
It doesn't even get the playlists anymore


[LIST=1]
[*]<item>
[*]<title>[COLOR yellow]Sky News[/COLOR][COLOR red][B] [YT][/B][/COLOR] [COLOR green]¤¤¤ [/COLOR]Oficial Channel[COLOR green] ¤¤¤[/COLOR]</title>
[*]<link>$doregex[makelist]</link>
[*]<regex>
[*]<name>makelist</name>
[*]<listrepeat><![CDATA[
[*] <title>[COLOR gold][makelist.param3][/COLOR]</title>
[*] <link>$doregex[makelist2]</link>
[*] <thumbnail>[makelist.param2]</thumbnail>
[*] <fanart>https://yt3.ggpht.com/S4XHNlwIVk7lqA3TZ5GT7WAwDwJtkxkLEA4i5wMjOkaNs3frg6zOfdf4fYtlq1LId67OHMZJvw=w2120-fcrop64=1,00005a57ffffa5a8-k-c0xffffffff-no-nd-rj</fanart>
[*]]]></listrepeat>
[*]<expres><![CDATA[#$pyFunction
[*]import re,urllib
[*]def GetLSProData(page_data, Cookie_Jar, m):#streamwatcher2013
[*] source=urllib.urlopen('https://www.youtube.com/c/SkyNews/channels').read()
[*] return re.findall('"gridPlaylistRenderer":{"playlistId":"([^"]+).+?"thumbnails":.{"url":"([^\?]+).+?"title":{"runs":.{"text":"([^"]+)',source)
[*]]]></expres>
[*]<page></page>
[*]</regex>
[*]<regex>
[*]<name>makelist2</name>
[*]<listrepeat><![CDATA[
[*] <title>[COLOR gold][makelist2.param1] [COLOR cyan]([makelist2.param2])[/COLOR]</title>
[*] <info>[makelist2.param3]</info>
[*] <utube>[makelist2.param4]</utube>
[*] <thumbnail>https://i.ytimg.com/vi/[makelist2.param4]/hqdefault.jpg</thumbnail>
[*]]]></listrepeat>
[*]<expres><![CDATA[#$pyFunction
[*]import re,requests
[*]def GetLSProData(page_data, Cookie_Jar, m):#streamwatcher2013
[*] count=1;pn=1;jdata=[]
[*] while pn <= int(count):
[*] source=requests.get('https://www.youtube.com/list_ajax?style=json&action_get_list=1&list=[makelist.param1]&index='+str(pn),headers={'user-agent':'Mozilla/5.0'}).json()
[*] jdata +=source.get('video');pn +=1
[*] channels=[]
[*] for j in jdata:
[*] title=j.get('title')
[*] vid=j.get('encrypted_id')
[*] inf=j.get('description')
[*] length=j.get('duration')
[*] channels.append((title,length,inf,vid))
[*] return channels
[*]]]></expres>
[*]<page></page>
[*]</regex>
[*]<thumbnail>https://yt3.ggpht.com/a/AATXAJzHMjrxBG0TA7AU1HoATeM9DZkmXtb-bmSTidOPL3Q=s100-c-k-c0xffffffff-no-rj-mo</thumbnail>
[*]<fanart>https://yt3.ggpht.com/S4XHNlwIVk7lqA3TZ5GT7WAwDwJtkxkLEA4i5wMjOkaNs3frg6zOfdf4fYtlq1LId67OHMZJvw=w2120-fcrop64=1,00005a57ffffa5a8-k-c0xffffffff-no-nd-rj</fanart>
[*]</item>
[/LIST]

 

Catmantx

Member
Nov 2, 2020
67
3
8
Sorry,
Rich (BB code):
<item>
<title>[COLOR yellow]Sky News[/COLOR][COLOR red]  [YT][/COLOR]  [COLOR green]¤¤¤  [/COLOR]Oficial Channel[COLOR green]  ¤¤¤[/COLOR]</title>
<link>$doregex[makelist]</link>
<regex>
<name>makelist</name>
<listrepeat><![CDATA[
  <title>[COLOR gold][makelist.param3][/COLOR]</title>
  <link>$doregex[makelist2]</link>
  <thumbnail>[makelist.param2]</thumbnail>
  <fanart>https://yt3.ggpht.com/S4XHNlwIVk7lqA3TZ5GT7WAwDwJtkxkLEA4i5wMjOkaNs3frg6zOfdf4fYtlq1LId67OHMZJvw=w2120-fcrop64=1,00005a57ffffa5a8-k-c0xffffffff-no-nd-rj</fanart>
]]></listrepeat>
<expres><![CDATA[#$pyFunction
import re,urllib
def GetLSProData(page_data, Cookie_Jar, m):#streamwatcher2013
  source=urllib.urlopen('https://www.youtube.com/c/SkyNews/channels').read()
  return re.findall('"gridPlaylistRenderer":{"playlistId":"([^"]+).+?"thumbnails":.{"url":"([^\?]+).+?"title":{"runs":.{"text":"([^"]+)',source)
]]></expres>
<page></page>
</regex>
<regex>
<name>makelist2</name>
<listrepeat><![CDATA[
  <title>[COLOR gold][makelist2.param1] [COLOR cyan]([makelist2.param2])[/COLOR]</title>
  <info>[makelist2.param3]</info>
  <utube>[makelist2.param4]</utube>
  <thumbnail>https://i.ytimg.com/vi/[makelist2.param4]/hqdefault.jpg</thumbnail>
]]></listrepeat>
<expres><![CDATA[#$pyFunction
import re,requests
def GetLSProData(page_data, Cookie_Jar, m):#streamwatcher2013
  count=1;pn=1;jdata=[]
  while pn <= int(count):
    source=requests.get('https://www.youtube.com/list_ajax?style=json&action_get_list=1&list=[makelist.param1]&index='+str(pn),headers={'user-agent':'Mozilla/5.0'}).json()
    jdata +=source.get('video');pn +=1
    channels=[]
    for j in jdata:
      title=j.get('title')
      vid=j.get('encrypted_id')
      inf=j.get('description')
      length=j.get('duration')
      channels.append((title,length,inf,vid))
  return channels
]]></expres>
<page></page>
</regex>
<thumbnail>https://yt3.ggpht.com/a/AATXAJzHMjrxBG0TA7AU1HoATeM9DZkmXtb-bmSTidOPL3Q=s100-c-k-c0xffffffff-no-rj-mo</thumbnail>
<fanart>https://yt3.ggpht.com/S4XHNlwIVk7lqA3TZ5GT7WAwDwJtkxkLEA4i5wMjOkaNs3frg6zOfdf4fYtlq1LId67OHMZJvw=w2120-fcrop64=1,00005a57ffffa5a8-k-c0xffffffff-no-nd-rj</fanart>
</item>
can't delete the previous post.
 

Beezlo

Member
Mar 17, 2016
33
0
6
HI,
need please regex for the both ..

hxxps://www.kanald.com.tr/diziler & hxxps://www.trt1.com.tr/tv/diziler

THANKS so much in Advance!!
 

streamwatcher2013

Well-known member
Oct 24, 2013
986
446
63
USA
@Beezlo

For TRT1 most the link from youttube

LSP XML:
<item>
<title>https://www.trt1.com.tr/tv/diziler</title>
<link>$doregex[makelist]</link>
<regex>
<name>makelist</name>
<listrepeat><![CDATA[
  <title>[COLOR gold][makelist.param2] [COLOR dodgerblue] ([makelist.param4])[/COLOR]</title>
  <link>$doregex[makelist2]</link>
  <thumbnail>https://www.trt1.com.tr[makelist.param1]</thumbnail>
]]></listrepeat>
<expres><![CDATA[#$pyFunction
import re
def GetLSProData(page_data,Cookie_Jar,m):
  source=re.findall('class="nav\s*navbar-nav">(?s)(.*?)<footer',page_data)[0]
  return re.findall('(?s)"img-responsive"\s*src="([^"]+)"\s*alt="([^"]+).+?href="([^"]+).+?"date-title">([^<]+)',source)
]]></expres>
<page>https://www.trt1.com.tr/tv/diziler</page>
</regex>
<regex>
<name>makelist2</name>
<listrepeat><![CDATA[
  <title>[COLOR gold][makelist2.param1] Bölüm[/COLOR]</title>
  <urlsolve>https://www.youtube.com/watch?v=$doregex[getUrl]</urlsolve>
  <thumbnail>https://www.trt1.com.tr/[makelist2.param2]</thumbnail>
]]></listrepeat>
<expres><![CDATA[#$pyFunction
import re,requests
def GetLSProData(page_data,Cookie_Jar,m):
  source=requests.get('https://www.trt1.com.tr[makelist.param3]',headers={'user-agent':'','referer':'https://www.trt1.com.tr/tv/diziler','accept':'*/*'}).content
  eps1='https://www.trt1.com.tr/'+re.findall('id="bolumler">[\w\W]+?(diziler/.+?)\d+',source)[0]+'1-bolum'
  source=requests.get(eps1,headers={'user-agent':'','host':'www.trt1.com.tr','accept':'*/*'}).content
  data=re.findall('</title>(?s)(.*?)class="detail-content">',source)[0]
  epsID=sorted(re.findall('(?:"og:url"\s*content|href).+?bolum/(\d+)',data), key=int)
  link=re.findall('(?:"og:url"\s*content|href).+?((?:arsiv|diziler).+bolum/[^"]+)',data)
  thumb=re.findall('(?:"og:image"\s*content|<img\s*src).+?(public/dm_videolar[^"]+)',data)
  return zip(epsID,thumb,link)
]]></expres>
<page></page>
</regex>
<regex>
<name>getUrl</name>
<expres><![CDATA[#$pyFunction
import re
def GetLSProData(page_data,Cookie_Jar,m):
  return re.findall('src=".+?youtube.com/embed/([^\?]+)',page_data)[0]
]]></expres>
<page>https://www.trt1.com.tr/[makelist2.param3]</page>
</regex>
<thumbnail>https://i4.hurimg.com/i/hurriyet/75/0x0/5ca312430f2544169477997c.jpg</thumbnail>
</item>
 
  • Like
Reactions: Beezlo and altug

streamwatcher2013

Well-known member
Oct 24, 2013
986
446
63
USA
Here is for the other:


LSP XML:
<item>
<title>https://www.kanald.com.tr/diziler/</title>
<link>$doregex[makelist]</link>
<regex>
<name>makelist</name>
<listrepeat><![CDATA[
  <title>[COLOR gold][makelist.param3][/COLOR]</title>
  <link>$doregex[makelist2]</link>
  <thumbnail>[makelist.param2]</thumbnail>
]]></listrepeat>
<expres><![CDATA[#$pyFunction
import re,requests
def GetLSProData(page_data,Cookie_Jar,m):#streamwatcher2013
  count=re.findall('>(\d+)</a></li></ul>',page_data)[0];pn=1;data=[]
  while pn <= int(count):
    source=requests.get('https://www.kanald.com.tr/diziler/arsiv?page='+str(pn),headers={'user=agent':'Mozilla/5.0','accept':'*/*'}).content
    list=re.findall('"listing">(?s)(.*?)</section',source)[0]
    data +=re.findall('class="item\s*col.+?href.+?/(\w[^"]+).+?src="([^"]+)"\s*alt="([^"]+)',list);pn +=1
  return data
]]></expres>
<page>https://www.kanald.com.tr/diziler/arsiv</page>
<referer>https://www.kanald.com.tr/diziler/</referer>
</regex>
<regex>
<name>makelist2</name>
<listrepeat><![CDATA[
  <title>[COLOR gold][makelist2.param4][/COLOR]</title>
  <inputstream>$doregex[getUrl]|user-agent=ipad&amp;origin=https://www.kanald.com.tr&amp;referer=https://www.kanald.com.tr/</inputstream>
  <thumbnail>[makelist2.param3]</thumbnail>
]]></listrepeat>
<expres><![CDATA[#$pyFunction
import re,requests
def GetLSProData(page_data,Cookie_Jar,m):#streamwatcher2013
  name='[makelist.param1]';url='https://www.kanald.com.tr/%s/bolumler'%name
  source=requests.get(url,headers={'user-agent':'','referer':'https://www.kanald.com.tr/%s/'%name,'accept':'*/*'}).content
  count=re.findall('>(\d+)</a></li></ul>',source)[0];pn=1;data=[]
  while pn <= int(count):
    source=requests.get('%s?page=%s'%(url,str(pn)),headers={'user=agent':'Mozilla/5.0','accept':'*/*'}).content
    list=re.findall('"listing-holder"(?s)(.*?)</section',source)[0]
    data +=re.findall('href.+?/(\w[^"]+).+?data-saved-id="([^"]+).+?src="([^"]+)"\s*alt="([^"]+)',list);pn +=1
  return data
]]></expres>
<page></page>
</regex>
<regex>
<name>getUrl</name>
<expres><![CDATA[#$pyFunction
import re,requests,json
def GetLSProData(page_data,Cookie_Jar,m):#streamwatcher2013
  url='https://www.kanald.com.tr/actions/media?id=[makelist2.param2]&p=1&pc=1'
  source=requests.get(url,headers={'user-agent':'','referer':'https://www.kanald.com.tr/[makelist2.param1]','accept':'*/*'}).content
  html,netloc=re.findall('"serviceUrl":"([^"]+)","securePath":"([^"]+)',source)[0]
  return '%s%s'%(html,netloc)
]]></expres>
<page>https://www.trt1.com.tr/[makelist2.param3]</page>
</regex>
</item>
 
  • Like
Reactions: Beezlo and altug
Thread starter Similar threads Forum Replies Date
G Scraper Development 0
J Scraper Development 1
O Addon Requests 5