while not keypressed (Posts about greener side)

Where are Those Gorgeous Facebook Notes?

Roman Yepishev — Sun, 07 Feb 2016 18:16:47 GMT

Have you searched for something lately and ended up reading a Facebook Notes post?

Yeah, me neither.

I gave Google a week to find my public note on facebook:

All hope is lost.

But it is really there:

This page really exists!

Guess what? No search engine other than Facebook itself can know about that post, because as soon as e.g. Google hits https://facebook.com/roman.yepishev it sees this:

Welcome to the walled garden.

There is no entry point to Facebook Notes. Will there be one? Maybe... No. Look at Facebook's robots.txt - it does not really want the search engines to look inside, but it does not mind if you end up with a direct link.

...
User-agent: Googlebot
Disallow: /ajax/
Disallow: /album.php
Disallow: /checkpoint/
Disallow: /contact_importer/
Disallow: /feeds/
Disallow: /file_download.php
Disallow: /hashtag/
Disallow: /l.php
Disallow: /live/
Disallow: /p.php
Disallow: /photo.php
Disallow: /photos.php
Disallow: /sharer/
Disallow: /topic/

...
User-agent: *
Disallow: /

So how can people read the notes?

There are two ways - they are either already browsing Facebook, and somebody on Facebook shared the note to their friends (less likely - they specifically search for the text in the note). Or, alternatively, if somebody outside of Facebook posted a direct link to the page.

But it does not have to be that way. You can find the only reference to my unique post hosted here from the sitemap.xml generated by the blog engine directly. And Google found it. Google is awesome at finding things:

It took Google almost a week, but it found the page. Because I want my posts to be found.

So please, publish your stuff on the open web (contrary to how it looks, Medium is not so open), somewhere the search gods (and mortal people) can actually find you.

From Zoho Vault to KeePassX

Roman Yepishev — Sat, 23 Jan 2016 15:43:36 GMT

I am constantly searching for the greener side. Sometimes I find one, sometimes I have to go back to the way I used to do things.

Zoho Vault is like LastPass - you provide your encryption key to the web page and it decrypts your data, you can also use an Android app. It is also completely free for personal use of a single account.

And so, having spent two months using Zoho Vault, I decided to migrate back to using KeePassX, though.

I found the web interface slightly annoying with the credential search taking a noticeable amount of time, password copy/paste depending on flash and constantly trying to mask the password that you've just unmasked to copy.

The firefox plugin was making the experience better (earlier versions would block your browser while you are logging in to Zoho, much improved now), but I again stopped feeling happy about off-loading all the keys to my kingdom to a third-party resource.

And then I also found that KeePassX has released a new major version, 2.0, making it compatible with the newer releases of KeePass Password Safe, a .NET application (which you can run under linux with mono).

KeePassX 2.0 is a substantial improvement over the 1.x version - the UI became simpler, the addition of custom tags made it easier to store related things together, and basically this is the future, so upgrade now.

But KeePassX also uses the new .kbdx format, and can only import the older .kdb one.

So, since I am usually going head-first without leaving an easy way back, I now had to get the data out of Zoho Vault and into KeePassX 2.

So I got the Zoho Export CSV, wrote a python script that would convert the entries to the .kdb format and then imported them into the new KeePassX 2 built in my Copr project.

Since there's currently no way to create kdbx files directly, I used the kppy module to generate the v1 database:

#!/usr/bin/python3 """Convert Zoho Vault Export CSV to KeePass KDB Usage: zohovault2keepass.py ZohoVault.csv keepass.kdb You will be prompted for the encryption password. """ import csv import collections import getpass import sys import kppy.database EXPORT_GROUP = 'Zoho Vault' ZohoVaultEntry = collections.namedtuple('ZohoVaultEntry', [ 'secret_name', 'description', 'secret_url', 'secret_data', 'notes', 'custom_data', 'tags', 'classification', 'favorite', 'chamber_name' ]) class ZohoVaultConverter(object): """Zoho Vault Converter""" def __init__(self): """Initialize converter""" self._database = None self._group = None def _prepare_db(self, path, password): """Prepare KDB database and group""" self._database = kppy.database.KPDBv1( filepath=path, password=password, new=True ) # This returns boolean, so we need to hunt for the group self._database.create_group(title=EXPORT_GROUP) for item in self._database.groups: if item.title == EXPORT_GROUP: self._group = item break if self._group is None: raise RuntimeError("Created group is not found") def convert(self, input_path, output_path, password): """Convert CSV-ish data exported from Zoho Vault to KDB""" self._prepare_db(output_path, password) entries = [] with open(input_path, 'r', encoding='utf-8') as filehandle: reader = csv.reader(filehandle) # fast-forward through the header next(reader) for row in [v for v in reader if v]: entries.append(self._process_row(row)) for entry in entries: entry["group"] = self._group self._database.create_entry(**entry) self._database.save() self._database.close() @staticmethod def _process_row(row): """Process Vault Entry""" if len(row) > 10: assert row[10] == '' entry = ZohoVaultEntry(*row[0:10]) secret_data = {} for line in [v.strip() for v in entry.secret_data.split('\n') if v != '']: key, value = line.split(':', 1) secret_data[key] = value comment = [] if entry.description.strip() != '': comment.append(entry.description) if entry.notes.strip() != '': comment.append(entry.notes) if entry.custom_data.strip() != '': comment.append(entry.custom_data) return { "title": entry.secret_name, "url": entry.secret_url, "username": secret_data.get("User Name", ''), "password": secret_data.get("Password", ''), "comment": '\n'.join(comment) } if __name__ == "__main__": if len(sys.argv) != 3: print(__doc__) sys.exit(1) converter = ZohoVaultConverter() master_password = getpass.getpass() converter.convert( sys.argv[1], sys.argv[2], master_password )

Usage:

$ pip3 install kppy
$ python3 zohovault2keepass.py ~/Downloads/zoho-export.csv zoho-export.kdb
Password: <master password>

And then use Database / Import KeePass 1 database menu entry.

Windows 10: I can't believe I am saying this (Part 1)

Roman Yepishev — Sat, 26 Dec 2015 19:20:28 GMT

This article originally appeared in Facebook Notes. And you won't find it through an external search engine, so copying it here.

For reasons beyond my understanding, I decided to switch to Microsoft Windows 10 for a brief time to see how greener the grass on the other side actually is. I left behind the Fedora install I had for the past year (and before that I had had Ubuntu since 2005), so I was mostly exposed to Gnome 3 environment, all the goodies the default Fedora repositories could provide, as well as some packages providing extra multimedia codecs.

So I got a copy of Windows 10 Pro and...

UI I almost saw on my laptop

Not really.

At first I had to fight a bit. It turns out that you can't reliably create a bootable USB drive of Windows 10 from a Linux machine. A direct write of a DVD image to the flashdrive got the Setup up and running, but it was unable to copy the files needed for an install. Unetbootin also failed.

So I had to go to a store and get a stack of blank DVDs for the first time in 5 years.

The installation went through, booted into a 1024x768 resolution on a widescreen display, but as I was creating an account in the Setup the driver got installed and the panel switched to a native resolution. This was as unexpected as a message from CUPS back in 2008 saying that my printer was plugged in on my Ubuntu installation. It just worked.

Unfortunately, the vendors need to sell new stuff to fuel their R&D and show profit, so I found that my late-2012 Lenovo E420 does not have a seal of approval from Lenovo to run Windows 10. And I was in for a surprise to find that the system downloaded and installed the drivers for power management (-50% fan noise), sound card (+100% signal), video adapter (+50% visual clarity), and the touchpad (which as it turns out actually supports pinch-to-zoom, whoa).

Now, remembering that the quirks tables exist in Linux drivers for a reason, I was pretty sure that something just pretends to be working.

The UI the system booted to was familiar. The Windows Start menu I used back in Windows XP days was gone, the replacement live tile thingie was not shocking (I still remember the feeling of OMG OMG THAT FLYING FULL SCREEN INTERNET EXPLORER SIGN IN YOUR FACE METRO when I booted a preview version of Windows 8).

Live Tiles - The Weather Tile was too live for me to take a good picture of

It turned out that for some reason the constant moving nature of these live tiles makes me uncomfortable. Whenever I saw some constant animation happening on my Ubuntu machine, I was almost always hearing elevated interest of the fan to keep the CPU/GPU temperatures down, so that was really a BAD THING. And that was not happening here. Probably that Power Management thing did what it was supposed to do.

Now, being still fresh after a reinstall I decided to see how well the muxless hybrid graphics is supported under the only system is was kind-of designed for. I have a hybrid Intel/AMD setup which I kept in integrated-only mode for the whole lifetime of my laptop, because that thing just does not work under Linux. The moment I tried switching either the kernel ABI changed, rendering AMD kernel driver useless, or Xorg changed an internal structure which the proprietary driver depended on, or something else equally frustrating happened.

So I went to the "BIOS Setup" and re-enabled Switchable Graphics.

The system booted into a native resolution and the fan started to make some noise. According to the process list, a Catalyst driver was in process of being installed (yay! Future! Drivers beamed to your device instead of hunting them on the world wide web). Then this window appeared, prompting me to recall the backward compatibility stories from Raymond Chen (the old new thing).

Windows 3.1 called and it wants its font back

This time it was a friendly message from an unknown application. So I rebooted.

The OS booted into a 1024x768 resolution and stayed this way for a day.

Now, if I were running a Linux distro, I would go look at the Xorg log, see what kernel modules are loaded… But the Event Viewer was of no use. No graphics-related errors during the boot, Intel Graphics and Media control panel item just did not start and there was no debug output I could use to find out what went wrong. The device manager was displaying both video cards as being up and happily running, and the system info also showed two devices. According to the system, it was all perfectly fine. According to my eyes, they wanted to pop.

Stretch marks all over the UI (Simulated)

Binging for the answer (nope, "Binging" is totally not a word I should have used to describe a process of searching at bing.com) I found that nobody really knows how to get the reason of the failure (no usable logs), so the driver reinstall was the only thing that should have helped. Thinking that the best way to get to a known good state would be to switch back to the integrated GPU, I ended up with exactly the same 1024x768 setup, but the screen took a minute (yes, around 60 seconds) to start displaying something after a cold boot, restart, or resume. I made it worse. That's when I started reading about the compatibility of the drivers, and found out that my system is not supported.

Reinstall.

Yup, with no logs to diagnose, no way back to a good state (system restore brought me to the same configuration), no actionable items in C:\Intel, which for some freaking reason contained log files for some media-related service as well as driver installation details, and no support from the vendor, I decided to give up and reinstall.

Back to a single Intel integrated GPU. Yay, my eyes are thanking me. A couple of days later I attempted the same trick, but explicitly installing the Windows 8 drivers for the GPU stack. I got the Switchable Graphics menu on every folder in Explorer, got the control panel displaying stuff, but the screen startup delay returned, now measuring about 30 seconds. Since a laptop would suspend/resume quite often, I decided to go back to Intel integrated GPU.

So now I could do some other things in the system, like launching Cortana...

The world is on fire

Unfortunately, no matter how I screamed at the microphone, the device picked up only some noise.

Remember the quirks I talked about before? Yup, it turns out that the driver Microsoft installed matches the device perfectly - the PCI vendor and device IDs are covered by the driver, but since manufacturers are honey badgers and they don't care about playing nice with others, the driver I had to download from Lenovo had some magic bits tweaked to get the mic working.

Oh, and then I had fun with the built-in Ricoh MMC/SD card reader too. You see, I had mixed success with it under Linux, with some cards being unreadable. It turned out that the driver is not included with Windows 10, but again, Windows 8 one is good enough. I was able to read all the cards. However, after a suspend/resume cycle I noticed that "System and compressed memory" process started chewing up CPU cycles like mad. Of course, no logs or direct documentation about the bug was discovered with Bing, so I went deeper.

Mark Russinovich wrote a wonderful "Process Explorer" - a fancy task manager for those who want to know as much as possible about the processes on their systems. It also showed an elevated usage in System. Having opened Properties / Threads I was told the offending thread is being busy spinning in risdxc64.sys, which was a Ricoh-provided driver. Disabling the card reader via the device manager immediately stopped the abnormal CPU usage.

Got you!

I would have avoided all these issues if I used a supported system, but problems happen with any software. It looks like there is a lot of guesswork and magic involved when something goes wrong.

At last I felt I accomplished something. I closed the lid of the laptop and went to sleep, looking forward to more challenges from the OS I hadn't touched in a long time.