I have trouble with a web crawler using the TOR network. It's misusing the gopher proxy on my page. I don't want to disable/block tor (that would be the easy way out). It's permanently changing user agents and ignoring robots.txt. It ignores HTTP status codes. I'm currently serving it 4MB binary garbage in form of Link. It sucked in about 40GB of data now, but it doesn't explode and keeps crawling. Any other idea about what to do with it?
Btw. there is a nice BBS reachable via telnet gopher.su 1234 (Extra Plus: It's running on a Playstation 3)
Phlog update: gopher://codevoid.de/0/posts/2019-04-27-manage-dotfiles-with-git.txt (https protocol works too)
@metamurks: Also, maybe switch to my https url: https://codevoid.de/tw.txt - txtnish can deal with gopher, but it would always fetch the full file even when nothing has changed (gopher doesn't support HEAD requests). So https might be faster.
@mdosch: Yes. I first thought gopher would be a good protocol for this purpose. But HTTP has the advantage, that you don't always need to fetch the whole file. You can do a HEAD and check for last-modified header.
@metamurks It's not really live. Check my 'File Storage' on gopher :-)
Migration successful. My page (http version) is now running on gopherproxy-c. Unfortunately not without functionality loss. If you used my site as search engine, you need to replace the smiley with "q". The redirect is now handled by nginx and nginx can not read emoji variables.
Evil Bobs brand new gopherproxy-c is double as fast as gopherproxy-go. Amazing! (migration is in progress)
You can now use https://codevoid.de/?q=%s as search provider in your browser. Requests that do not start with gopher:// will be forwarded to the non-JS version of duckduckgo. Gopher will be shown directly. I'm not logging anything. But you must be aware that your search terms would touch my server first.
I got my new bag from #getdigital gopher://codevoid.de/I/p/IMG_5271.png
Worst IAP ever. 15sec w/o ads? really? gopher://codevoid.de/I/p/IMG_5270.png
The workflow app on iOS is magic. I now have a button that asks me to select a picture, then converts it to png, resizes it, strips the metadata, scps it to my jumphost, scps it further to my gopher jail and into my paste directory, constructs the http proxy URL and opens it in safari. All without user-interaction. Now I can share my mobile life with you guys! Prepare for cat pictures!
This tweet was written on an iphone. (via http proxy to gopher to txtnish)
I've started to manage my gopher content with RCS. Together with rcsvers.vim it's a no-brainer. It's pretty nice to have an automatic version created whenever vim is being closed.
Use the x-use-gopher header on your http proxies.. "curl -sI https://codevoid.de | grep ^x-u" bitreich.org, r-36.net, taz.de are already there. #gopher
...and I've also added my last tweets to my gopher hole. I don't know why, but this makes me happy.
txtnish people can follow me via gopher! @ gopher://codevoid.de/0/tw.txt
Timeline Sandbox