build.panel download from psid website failing

[shdoron1]

Hi there,

I believe with recent changes to the UMich website, build.panel() no longer successfully connects? After inputting my user and password I get the error:

Error in get.psid(family[i, "file"], name = paste0(datadir, "FAM", family[i, :
no longer logged in

This error was also encountered by a previous user trying to access the data behind a firewall in China. I tried disabling my local firewall (I'm in the US) an it did not resolve the issue. Prior, I had no problems accessing the data this way.

As a side note, if this method is no longer possible -- since build.panel()'s datadir argument relies on downloaded psid data in RDS/ RDA format, is there a recommended procedure for converting the downloaded PSID .txt files into RDS/ RDA?

Many thanks,
Doron

[floswald]

Hi @shdoron1 ,
thanks for that report. can you log in on the psid website with your credentials? like here

https://simba.isr.umich.edu/U/settings.aspx

[shdoron1]

Hi @floswald,

Yes, I have no problem logging in and downloading the data directly from PSID (either through a cart or direct Zip download).

Best,
D

[jdebacker]

@floswald Same issue for me (and I also tested my credentials) on the PSID website.

I haven't fully traced this out, but I think it might be worth looking into the conditional on line 113 of makeids.r. The search here looks for Logout in the html string to confirm one is logged in. It's not there -- but could that just be a change to the html -- ie., the logout button has moved on the PSID page? Is there another condition that could be checked to confirm the credentials were successfully authenticated?

[floswald]

good lead - I have no idea, i copied this part from upstream on @ajdamico's lodown package which is no longer maintained. see here . not sure i'll have time soon for this

[grischard]

Simba now also has a cloudflare captcha in front of it, making downloads more complicated. It's very unfortunate that there's no API way of downloading this data; it's not a huge amount of data in the end, and the portal is cumbersome.

[floswald]

hmm...well that's an issue yes. not sure what a solution could be here. I guess people have to resort to manually download again like in the good old days.