The detection of the encoding in XML TV file takes ages

[thombet]

I ran into an issue when using this add-on with a huge XML TV file: 207902 lines!

The regex at main.py@2927 didn't detect the encoding at the top of the file because the value was enclosed in single quotes instead of double quotes. Then it fell back to using chardet.detect() which basically never ended on my device (keep on displaying Copying xmltv file for several minutes on my Raspberry PI 3).

[thombet]

I created the pull request #36 to solve this by:

• updating the regex to support also the single quotes
• using only the first 50,000 characters in chardet.detect() instead of the compelete file if the file has more than 50,000 characters

Note: for improving chardet.detect() I initially tried what was suggested in chardet docs (i.e. incremental detection)) but the results were the same since the detection did not end before the end of the file... Maybe it is because the file is not actual text but XML code.
Some data from the tests I run on my PC (which is much faster than my RasPI 3):

• chardet.detect() on the full file ==> 1 min 55
• chardet.detect() on the first 50,000 characters ==> 1 second
• incremental detection by line ==> 2 min 30
• incremental detection by range of 1000 characters ==> 1 min 50

[Dobi-Dev]

Please note that to avoid some conflicts, I have integrated your PR in mine : #39.
I don't know if it was a good idea (but I would have to have your fix :D)

[thombet]

Please note that to avoid some conflicts, I have integrated your PR in mine : #38.
I don't know if it was a good idea (but I would have to have your fix :D)

Thanks for the info @Dobi-Dev! And it seems you also did some nice improvements that I thought of when implementing my change (like creating functions to decrease the amount of duplicate code): nice work :)

I don't know if any of this will be merged though since this repository was not updated in the last year...

One side note off-topic: the feature "record and play" (that mimics a timeshifting feature) does not work on my side. I would like to investigate this and propose a fix if I can. Did you notice the same thing or is it working on your side?

[Dobi-Dev]

Yes, I am wondering if it will be merged too. If it is not merged, I think we should continue working on another repository.

Off-topic : About the record and play, no I didn't notice that but I will check that now.
Edit : If it was the issue on my repository, it is fixed.

[thombet]

Yes, I am wondering if it will be merged too. If it is not merged, I think we should continue working on another repository.

I agree, maybe we should merge these versions in one of the forks. Among the 12 forks on this repo, they are all very old (except yours and mine).

@primaeval do you still have time to maintain this repository?

Off-topic : About the record and play, no I didn't notice that but I will check that now.
Edit : If it was the issue on my repository, it is fixed.

I didn't try your fork but for sure it is not working on mine. I will investigate the problem on my side. Thanks for checking :)

[Dobi-Dev]

Have you tried to "record and play" on several channels ?

[thombet]

(Sorry for the late reply)

Have you tried to "record and play" on several channels ?

Yes and the result is the same. I have the problem on RPI3 with OSMC. Which OS do you use?

[Dobi-Dev]

I am on my windows computer (I am using this plugin to get tv : https://catch-up-tv-and-more.github.io/installation/ ). I will try asap on my RPI4 + LibreElec.

[Dobi-Dev]

I have checked on my RPI4 and with the version of my repository and it was working.

[thombet]

I have checked on my RPI4 and with the version of my repository and it was working.

I am also using Catch-up TV & More. Maybe the RPI3 is not powerful enough for this feature (event though I don't understand why I can record, play but not use the record and play feature). I don't have the time to debug this for the moment I may investigate some day.
Thank you for checking on your side :)