npm link: What the first `npm link` really does and how it can also be used

Published 2018-11-28, 13:39

There are loads of articles about npm link, how it works (npm link in library dir, then npm link library in project that should use that local library as a dependency) and what it can be used for (linking to a local folder for a dependency instead of having a npm downloaded version in node_modules).

But most skip over or only scrape what the first of the two steps really does – and how that can be used besides the default use case.

TLDR: npm link not only creates a symlink to the library in the global node_modules (that can be used in the second step of npm link ...), but also makes the library main script or binary accessible via a global npm module, similar to what happens when you npm install -g something.

The details:

When you run npm link as the first step of the two-step linking process, you create a „symlink in the global folder„.

Let’s follow the individual sub steps:

C:\Projects\Cordova
λ npm list -g --depth=0 
C:\Program Files\nodejs
+-- npm@6.4.1

We start in a specific folder (C:\Projects\Cordova in my example) and check our globally installed packages: We have nothing besides npm installed on this machine.

C:\Projects\Cordova              
λ cd cordova-paramedic 
 
C:\Projects\Cordova\cordova-paramedic
λ npm link
removed 1 package in 3.649s
C:\Program Files\nodejs\cordova-paramedic -> C:\Program Files\nodejs\node_modules\cordova-paramedic\main.js
C:\Program Files\nodejs\node_modules\cordova-paramedic -> C:\Projects\Cordova\cordova-paramedic

Now we switch the directory of our library that we want to link, cordova-paramedic here. We run npm link which creates two symlinks:

  1. C:\Program Files\nodejs\cordova-paramedic -> C:\Program Files\nodejs\node_modules\cordova-paramedic\main.js
  2. C:\Program Files\nodejs\node_modules\cordova-paramedic -> C:\Projects\Cordova\cordova-paramedic

The first one is a symlink for the „binary“, the main script, of our library.
The second is a symlink for the whole library into the global node_modules.
(C:\Program Files\nodejs\ is our global node/npm folder on this machine)

That first symlink is not mentioned in most of the articles about npm link on the web as the example libraries being linked don’t include a main script in their package.json.

λ npm list -g --depth=0
C:\Program Files\nodejs
+-- cordova-paramedic@0.6.0-dev -> C:\Projects\Cordova\cordova-paramedic
`-- npm@6.4.1

When we check the globally installed modules again, we see that our library was added to that list.

So not only can we now use npm link cordova-paramedic, the second command in the npm link two step process, to add this library to any project from our local checkout – but we can also use it as a global command cordova-paramedic!

C:\Projects\Cordova
λ cordova-paramedic
Error missing args.
...

So using npm link you can not only link local development versions of libraries to use in another project, but you can also create a global command that uses a local version of your command code. This is very useful when developing such a library.

Topic(s): Development No comments - :(

Android Emulator: PANIC: Broken AVD system path. Check your ANDROID_SDK_ROOT value

Published 2018-11-16, 17:38

Debugging and fixing a broken Android SDK setup is always „fun“ – especially on a CI environment where you don’t control the actual installation.

It gets worse when all the StackOverflow answers and other Google results for the error you are getting are basically „I executed some slightly related, but otherwise random commands, and after that it worked“.

In my specific case I was getting this error message when trying to start an Android Emulator:

PANIC: Broken AVD system path. Check your ANDROID_SDK_ROOT value

It was really hard to find out what „AVD system path“ actually meant and how my ANRDOID_SDK_ROOT could be connected to that.

So to maybe make this search a bit easier for future people having this problem, I write a solution down here:

Continue reading Android Emulator: PANIC: Broken AVD system path. Check your ANDROID_SDK_ROOT value…
Topic(s): Development, Link No comments - :(

How to create an empty file in cmd.exe / Windows Command Line

Published 2018-11-14, 16:45

copy NUL EMptyFile.txt
copy /b NUL EmptyFile.txt

 https://stackoverflow.com/a/1702790/252627

Topic(s): Kram No comments - :(

Azure Devops Pipelines: Software available on Microsoft hosted agents

Published 2018-11-14, 15:27

https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=vsts&tabs=yaml#software contains links to READMEs on GitHub that list the available software

Topic(s): Development, Link No comments - :(

Azure Devops Pipelines: Install and start an Android Emulator

Published 2018-11-14, 15:17

#!/usr/bin/env bash

echo "y" | $ANDROID_HOME/tools/bin/sdkmanager --install 'system-images;android-27;google_apis;x86'

$ANDROID_HOME/platform-tools/adb devices

echo "no" | $ANDROID_HOME/tools/bin/avdmanager create avd -n test_android_emulator -k 'system-images;android-27;google_apis;x86' --force

nohup $ANDROID_HOME/emulator/emulator -avd test_android_emulator -no-snapshot > /dev/null 2>&1 & $ANDROID_HOME/platform-tools/adb wait-for-device shell 'while [[ -z $(getprop sys.boot_completed | tr -d '\r') ]]; do sleep 1; done; input keyevent 82'

echo "Emulator started" 

https://developercommunity.visualstudio.com/content/problem/340547/azure-devops-unable-to-run-the-android-emulator-on.html

https://github.com/MicrosoftDocs/vsts-docs/issues/1677#issuecomment-433858827

Topic(s): Link No comments - :(

How to comment in cmd.exe / Windows command line

Published 2018-11-14, 15:02

REM this is a comment

or

:: another comment

(Second one only works sometimes, see comment)

Topic(s): Link, Notiz, Technik 2 comments - :)

Results of an Amazon.de „Datenauskunft“ / „Data Subject Access Request“

Published 2018-09-18, 17:14

On 2018-07-20 I requested a „Datenauskunft“/“Data Subject Access Request“ from Amazon.de where I had an account since 2001. I wanted to document both the process and the results, as I couldn’t find anything online back when I requested this myself


To request a Datenauskunft from Amazon yourself you have to go to amazon.de/kontakt, then select „Digitale Inhalte und Services“, choose „Datenauskunft beantragen“ („Request your data“) in the first dropdown, „Datenauskunft für eine spätere Zusendung beantragen“ („Request data file be sent at a later date“) in the second and „Daten aus allen Kategorien anfordern“ („Request All Your Data“) in the third and last one.

Clicking „E-Mail“ will populate the contact form with some default text that you can just submit after adding your full name to the body text.


Shortly after I submitted the form, I got an email that I had reply to to confirm that I, the owner of my account email address, had indeed requested this data.

(I had to jump through a few hoops here as I use catch all email adresses @janpiotrowski.de for login, but replied from my usual email address which of course their system didn’t like. But as this is for verification of my request, I was happy to comply.)


I got a first response with data ~20 days later on 2018-08-09, which included a download link to a file name `Piotrowski.zip` (which was 138 KB of size).

The email also included the following admission:

Da wir für Ihre Anfrage Daten aus mehreren Bereichen zusammenstellen, gibt es einige zusätzliche Daten, die wir derzeit noch sammeln. Sobald Sie auch diese Daten herunterladen können, werden wir Sie entsprechend informieren.

The data file’s content:

1. DSAR_-_Jan_Piotrowski_-_Kontakthistorie.docx
2. DSAR_Anfrage_-_Jan_Piotrowski_-_Kontoinformationen.docx
3. Mein_Profil.xlsx

The `.docx` Word files contained 1) a history of (some recent) written communication with Amazon (including my data request of course), 2) all my current and past shipping addresses, payment methods, failed payments (just one…), watch list(s) content, vouchers used (lots), warranty claims (none), wish list(s) content, subscriptions (prime trial) and some information about Amazon Drive (never used) – really everything I could probably have gotten from the Amazon website myself after logging into my account there.

The `Mein_Profil.xlsx` just contained a link to www.amazon.de/profile.


Roughly one month later on 2018-09-17 I received another email from Amazon, telling me that now all data had been collected and could be provided to me.

Again it contained a download link to a file, this time called Jan Piotrowski.zip, now grown to 1043 KB (10.9 MB unzipped!).

This is its folder and file structure:

|   
+---- Ihr Kundenkonto -
|       DSAR - Piotrowski, Jan - Kontakthistorie.docx
|       DSAR_Anfrage - Piotrowski, Jan  - Kontoinformationen.docx
|       
+---Alexa
|   \---Alexa_0032...ea17a
|       +---Communication - Messages
|       |   \---Conversations
|       |           Conversations.html
|       |           
|       +---Communication - Preferences
|       |       alexa_comms_preferences.html
|       |       
|       +---Lists
|       |       Lists.csv
|       |       
|       +---Preferences
|       |       Preferences.json
|       |       
|       \---Routines
|               Routines.json
|               
+---Appstore
|       PurchaseDownloadInstall_Appstore.csv
|       
+---CloudDrive
|       Piotrowski_Jan_Account_Info.csv
|       Piotrowski_Jan_Client_Events.csv
|       Piotrowski_Jan_Device_Info.csv
|       Piotrowski_Jan_Node_Metadata.csv
|       
+---Community Profile
|       Mein Profil.xlsx
|       
+---Fire TV
|   |   3poptout_000.csv
|   |   appusage_000.csv
|   |   deviceusage_000.csv
|   |   marketingoptout_000.csv
|   |   registration.csv
|   |   
|   \---FireTV_Piotrowski_Jan
|       \---FireTV_Piotrowski_Jan
|               2018-01-23.json
|               2018-01-26.json
|               ...
|               FireTv-Glossary.csv
|               
+---Kindle
|   +---Geräte
|   |       appusage_000.csv
|   |       deviceusage_000.csv
|   |       marketingoptout_000.csv
|   |       registration.csv
|   |       
|   \---Inhalte
|       |   householdsharing.csv
|       |   KindleReadingActions.csv
|       |   ReadingSessions.csv
|       |   whispersync.csv
|       |   
|       \---digitalcontentownership
|               DigitalMusicTrack_B00...AG.json
|               DigitalMusicTrack_B00...DS.json
|               ...
|               KindleEBookSample_B00...W0.json
|               KindleEBookSample_B00...WA.json
|               ...
|               KindleEBook_B00...L0.json
|               KindleEBook_B00...W0.json
|               ...
|               KindlePDoc_2UEXX...TSOH.json
|               KindlePDoc_3753F...0766F1E.json
|               ...
|               MobileApp_B00...BO.json
|               MobileApp_B00...1W.json
|               ...
|               
+---Prime Music
|       DSAR_Jan_Piotrowski_Meine_Musikbibliothek.csv
|       
\---Prime Video
        Genderlanguage.csv
        Locationdata.csv
        Viewcounts.csv
        Viewinghistory.csv

As you can see this new package includes the same files as the first package in `/- Ihr Kundenkonto -` and `/Community Profile`, but also adds several other folders with my real usage data from Amazon:

The easiest way to look at the data was to just upload the whole folder to Google Drive. The previews of the `.csv` files is decent enough to get a quick overview.

Opening some of the files with Google Spreadsheet then enabled me to better format the available data and add columns to e.g. parse strange date format columns, sort by date columns and so on.


After a first look through the data files, they seem to be a pretty solid representation of what I expected to get from Amazon.

Some „logs“ are a bit short, only showing data from the last few months which is surprising, after I looked into those more I will possibly check with Amazon if there isn’t more that they just didn’t include by default.

I’m pretty happy with the time it took Amazon to compile this as well. Instant response and download or even self-serve would have been great, but I understand how much work it might be to compile all this data if there are not automated processes in place (yet). I expect this to get faster and better in the future.

Topic(s): Kram, Link, Notiz No comments - :(

Google Spreadsheet: Extract proper dates from funny date strings

Published 2018-09-18, 13:11

Recently Amazon sent me a „Datenauskunft“ that included a .csv of my Kindle’s Whispersync activity with a column containing dates, but in this funny format:

FriApr1412:21:01UTC2017
WedApr2610:43:54UTC2017
WedApr2611:30:41UTC2017
WedApr2611:30:40UTC2017
FriMay1119:26:44UTC2012
TueNov0819:47:36UTC2016
SunJan1022:12:49UTC2016

To get a proper date out of this, I built this formula to be used in a new column:

=DATE(RIGHT(C2;4);MATCH(MID(LEFT(C2;8);4;3);{"Jan";"Feb";"Mar";"Apr";"May";"Jun";"Jul";"Aug";"Sep";"Oct";"Nov";"Dec"};0);RIGHT(LEFT(C2;8);2))

`C2` being the column with the date here, this first extracts the year (4 characters from the right), then converts a 3 character month name to a number (which it got from extracting char 4-6 from the first 8 chars of the string), and finally the date which are the last 2 chars of the first 8 char block. The time is not needed here and discarded.


More variants:

WedApr2611:30:41UTC2017=DATE(RIGHT(C2;4);MATCH(MID(LEFT(C2;8);4;3);{"Jan";"Feb";"Mar";"Apr";"May";"Jun";"Jul";"Aug";"Sep";"Oct";"Nov";"Dec"};0);RIGHT(LEFT(C2;8);2))
31-DEC-2015 17:26:15=DATE(LEFT(RIGHT(F2;13);4);MATCH(MID(F2;4;3);{"Jan";"Feb";"Mar";"Apr";"May";"Jun";"Jul";"Aug";"Sep";"Oct";"Nov";"Dec"};0);LEFT(F2;2))
Sun May 08 09:47:24 UTC 2011=DATE(RIGHT(D2;4);MATCH(MID(LEFT(D2;8);5;3);{"Jan";"Feb";"Mar";"Apr";"May";"Jun";"Jul";"Aug";"Sep";"Oct";"Nov";"Dec"};0);RIGHT(LEFT(D2;10);2))
09/May/2011 11:49:10 UTC=DATE(MID(D2;8;4);MATCH(MID(D2;4;3);{"Jan";"Feb";"Mar";"Apr";"May";"Jun";"Jul";"Aug";"Sep";"Oct";"Nov";"Dec"};0);LEFT(D2;2))


Topic(s): Notiz 1 single comment - :/

Berlin focused Facebook Flohmarkt groups

Published 2018-09-14, 12:40

Any important ones missing?

Topic(s): Link No comments - :(

„Turn off“ output truncation of RSpec output

Published 2018-08-27, 19:16

Use this code to increase the output length before RSpec trancates output like e.g. `expected` and `got` in test failure results:

RSpec::Support::ObjectFormatter.default_instance.max_formatted_output_length = 1024

Topic(s): Technik No comments - :(

16 queries. 0,281 seconds.