I saw a post on LinkedIn about a platform called HarperDB.
It’s billed as a distributed database, cache, and pub/sub engine rolled into one. That sounded interesting enough to try out.
HarperDB is a single-system platform that combines a SQL-like API, REST endpoints, and MQTT messaging on top of a document-style data model. It runs locally or in the cloud, and the setup takes less than a minute.
If you open any of the popular music streaming services, ask it to play Johann Sebastian Bach, there’s about a 100% chance that it’s going to start with the Bach unaccompanied cello Suite No. 1 performed by Yo-Yo Ma. It’s a great piece, of course, but not what I want to hear every time. If you skip forward through that piece, you’ll likely land on Glenn Gould mumbling along to his performance of The Goldberg Variations on piano.
To get around this, I started telling Siri or Alexa to play a random piece, using the Bach catalog number, such as “Alexa, play Bach’s BWV xxx” where xxx is a random number. (For example, the Goldberg Variations is BWV 988 and the six Cello Suites are BWV 1007–1012.)
The BWV (Bach-Werke-Verzeichnis, or Bach Works Catalog) numbering system, created by musicologist Wolfgang Schmieder in 1950, isn’t strictly chronological. Instead, it organizes Bach’s compositions thematically by genre—starting with cantatas, followed by choral works, keyboard music, and so forth.
It’s an imperfect system, though, as Bach pieces get discovered from time to time. If, for example, they discovered a seventh cello suite, it wouldn’t get to be BWV 989, as that number is occupied by his “Aria variata alla maniera italiana”, so it gets tacked on to the end of the catalog. (I’d prefer they go with something like BWV 988.001 to get the piece in the right spot, but whatever…)
Recently, I stumbled upon a Spotify playlist containing all known cataloged works by Johann Sebastian Bach, from BWV 1 to BWV 1128. Intrigued by this, I decided to embark on the arduous task of listening through them sequentially.
Anyway, here’s the playlist for the whole catalog. If you’re not up for a sequential run through the 136 hours of the whole lot, you can put it on random and play any of the 3,196 pieces.
Distributed systems are fundamental to modern computing, but they come with inherent trade-offs. The CAP theorem, introduced by Eric Brewer, provides a framework for understanding these trade-offs by defining three key properties: Consistency, Availability, and Partition Tolerance. In this article, we will explore CAP theorem step by step, starting from its basic definition to its deeper implications in distributed system design.
Imagine an online banking system with multiple servers across different regions. If a customer deposits money in one branch, that update must reflect accurately across all servers to prevent double spending. However, if a network failure occurs between data centers, the system faces a dilemma:
The CAP theorem provides a framework to understand and resolve such trade-offs in distributed system design.
The CAP theorem states that in a distributed data system, it is impossible to achieve all three of the following simultaneously:
In practice, distributed systems must make trade-offs between these properties, choosing to optimize for two at the expense of the third.
Consistency ensures that all nodes return the same data at any given time. If a write is performed on one node, all subsequent reads on any node should reflect that write.
Example: A traditional relational database (e.g., PostgreSQL) ensures strict consistency by using distributed transactions.
Availability ensures that every request to the system receives a response, even if some nodes are unavailable.
Example: A DNS system is highly available, meaning queries always receive responses, even if some servers are down.
Partition tolerance means that the system continues to operate even when network failures occur, causing communication issues between nodes.
Example: A global system like Apache Cassandra remains operational even if some nodes lose connectivity due to network failures.
Since no distributed system can achieve all three properties, they typically fall into one of the following categories:
When designing a distributed system, understanding CAP helps in making informed trade-offs:
Many modern distributed databases adopt a flexible consistency model rather than strictly following CAP trade-offs:
For a deeper understanding of CAP theorem, check out:
A distributed system needs a reliable mechanism for reaching consensus across multiple nodes. The RAFT algorithm is one such consensus algorithm. It was designed to be easier to understand than Paxos (an earlier, more complex protocol for providing consensus in distributed networks,) while maintaining strong fault tolerance.
RAFT is widely used in distributed systems that require strong consistency. In a distributed environment, multiple nodes work together to store and process data reliably. However, ensuring that all nodes agree on the same state at any given time is a challenge. RAFT helps solve this problem by ensuring that commands are consistently applied across all nodes, preventing conflicts and maintaining data integrity. Unlike traditional databases that rely on a single primary instance for writes, distributed systems using RAFT elect a leader that coordinates operations, ensuring that updates are applied in the same order across all nodes. This makes RAFT a crucial component for building fault-tolerant, highly available distributed applications.
RAFT (Reliable, Replicated, and Fault-Tolerant) is a consensus algorithm used to manage a replicated log in distributed systems. It ensures that multiple nodes in a distributed system agree on the same sequence of commands, maintaining consistency even in the face of network failures.
(For comparison, another well-known consensus algorithm is Paxos, which is more complex but serves a similar purpose. You can read more about it here.)
RAFT is used in distributed databases, coordination services, and in-memory data grids like Hazelcast to ensure consistency.
RAFT divides nodes into three roles:
At any given time, there is at most one leader, while all other nodes function as followers.
When a RAFT cluster starts, or if the leader fails, an election process takes place:
Node A (Follower) -> Times out -> Becomes Candidate
Node A requests votes from Nodes B & C
Nodes B & C vote for A (majority wins)
Node A becomes the LeaderThis mechanism ensures a stable leadership structure even during network failures.
Once a leader is established, it manages log replication:
A log in RAFT is an append-only structure that stores client requests (commands). The leader ensures that all followers maintain an identical sequence of logs so that they reach the same state.
Client -> Sends "Write X=10" to Leader
Leader -> Appends "Write X=10" to log
Leader -> Sends entry to Followers
Majority acknowledges -> Entry is committedThis ensures all nodes eventually apply the same commands in order.
| Log Index | Term | Command |
|---|---|---|
| 1 | 1 | Write X = 5 |
| 2 | 2 | Write Y = 20 |
| 3 | 2 | Write X = 10 |
Logs serve as the foundation for fault tolerance in RAFT, ensuring that even if a node fails, the system can recover and maintain a consistent state.
RAFT ensures that failures do not cause inconsistencies by following strict rules:
This design prevents split-brain scenarios and guarantees system integrity.
Many distributed systems, including Hazelcast, leverage RAFT for high availability and fault tolerance. Hazelcast’s CP Subsystem implements RAFT to ensure data consistency in distributed environments.
For a deeper dive into RAFT, check out:
Secure Shell (SSH) is a fundamental tool for secure remote access. However, relying solely on SSH keys can still pose a risk if a key is compromised. Adding Two-Factor Authentication (2FA) enhances security by requiring an additional verification step, reducing unauthorized access risks.
Enhanced Security – 2FA ensures that even if an SSH key is stolen, an attacker cannot log in without the second factor.
Reduced Risk – The requirement of a secondary authentication method, such as a time-based one-time password (TOTP), significantly decreases unauthorized access risks.
Compliance – Many security frameworks and regulations, including , recommend or mandate 2FA for secure system access.
To enable 2FA, install a module like or on your server:
sudo apt install libpam-google-authenticator # For Debian-based systems
Edit /etc/ssh/sshd_config to enable challenge-response authentication:
ChallengeResponseAuthentication yes
AuthenticationMethods publickey,password publickey,keyboard-interactive
Then, edit /etc/pam.d/sshd to include:
auth required pam_google_authenticator.so
Restart the SSH service for changes to take effect:
sudo systemctl restart ssh
Run the Google Authenticator setup:
google-authenticator
Follow the prompts to generate a QR code and store the recovery codes safely. Use an authenticator app like or to scan the QR code.
Attempt to log in using your SSH key first, followed by the TOTP code from your authentication app:
ssh user@your-server
If configured correctly, you will be prompted for your SSH key authentication and then asked for the 2FA code.
By implementing 2FA alongside SSH keys, you significantly strengthen remote access security. It ensures that only authorized users with both factors can gain access, reducing the risk of breaches. Additionally, using 2FA aligns with modern security best practices and compliance standards, offering peace of mind.
For further reading, check out:
Secure Shell (SSH) authentication relies on key pairs to provide secure access to remote systems. Various key algorithms offer different levels of security, performance, and compatibility. In this article, we’ll compare four commonly used SSH key algorithms: RSA, DSA, ECDSA, and Ed25519.
RSA (Rivest-Shamir-Adleman) is one of the most widely supported and trusted SSH key algorithms. It provides strong security, especially with key lengths of 2048 bits or more.
To create an RSA key pair, use the following command:
ssh-keygen -t rsa -b 2048 -C "your_email@example.com"
This command generates a 2048-bit RSA key pair and associates it with your email address.
Modify the SSH server configuration file (/etc/ssh/sshd_config) with these settings:
PubkeyAuthentication yes
RSAAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
RSA is a robust and compatible choice for most SSH authentication needs.
DSA (Digital Signature Algorithm) was once a common choice but is now considered outdated due to its fixed 1024-bit key size, which is no longer considered sufficiently secure.
ssh-keygen -t dsa -b 1024 -C "your_email@example.com"
Since DSA keys are limited to 1024 bits, they are less secure than other algorithms.
Add the following lines to /etc/ssh/sshd_config:
PubkeyAuthentication yes
DSAAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
While DSA offers fast key generation, its security limitations make it less favorable for modern deployments.
ECDSA (Elliptic Curve Digital Signature Algorithm) is designed for better performance with smaller key sizes compared to RSA while maintaining strong security.
ssh-keygen -t ecdsa -b 256 -C "your_email@example.com"
ECDSA supports key sizes of 256, 384, and 521 bits, with 256-bit ECDSA offering a good balance of security and performance.
Modify /etc/ssh/sshd_config to include:
PubkeyAuthentication yes
While ECDSA is more efficient than RSA, its security depends on the choice of an elliptic curve, and some concerns exist regarding its implementation.
Ed25519 is a modern elliptic curve algorithm designed for high security, fast performance, and resistance to side-channel attacks.
ssh-keygen -t ed25519 -C "your_email@example.com"
This command generates a compact and efficient key pair suitable for modern SSH authentication.
Edit /etc/ssh/sshd_config and ensure the following is present:
PubkeyAuthentication yes
Ed25519 is increasingly favored for its speed, security, and compact key sizes.
When selecting an SSH key algorithm, consider the following factors:
| Algorithm | Key Size | Security | Performance | Compatibility |
|---|---|---|---|---|
| RSA | 2048+ | Strong | Moderate | Widely Supported |
| DSA | 1024 | Weak | Fast | Limited Support |
| ECDSA | 256+ | Strong | High | Moderate |
| Ed25519 | 256 | Very Strong | Very High | Growing Adoption |
For general use, RSA (2048-bit or higher) is a reliable and widely supported choice.
For better performance, Ed25519 provides strong security with fast authentication.
For constrained environments, ECDSA offers efficiency but requires careful implementation.
Avoid DSA unless compatibility reasons require it, as it is outdated and less secure.
SSH is how you log in to remote servers and devices.
Once upon a time, people used
telnet, but telnet wasn’t encrypted, so it was replaced with the “Secure Shell”, orssh.
Like telnet, it’s possible to log in using your username and password, but, if you’re still relying on passwords in this day and age, you’re leaving yourself vulnerable. SSH keys are far superior in terms of security and once you have them set up, they’re much more convenient.
In this article, we’ll talk about why SSH keys are essential, how to create them, and best practices for ensuring their security.
To create SSH keys, follow these steps:
ssh-keygen command to generate a key pair. You can specify the key length and encryption algorithm. For example, to create a 4096-bit RSA key:ssh-keygen -t rsa -b 4096 -C "your_email@example.com"
~/.ssh/id_rsa, and set appropriate permissions:chmod 600 ~/.ssh/id_rsa
~/.ssh/authorized_keys file:ssh-copy-id user@remote_server
Don’t compromise on security; follow these steps to make sure your SSH keys are secure and well-managed.
If you are a moderate to heavy user of the command line in Linux or OS X, you’ll eventually build up a set of custom commands and shortcuts that you’ve written that help streamline your workflow. These are known as “aliases.”
Let’s say you frequently find yourself searching your shell history for a command you used before.
To display your command history, you simply type `history`in your shell, and every command you’ve typed streams by.
This is useful, of course, but most likely you want to know about a particular command you typed, so you might use shell redirection to pipe the output to another command, in this case, `grep` to find lines in history that are interesting to you. In this example, we’re looking for commands we’ve typed that involved the word “projects”:
$ history | grep projects
9627 cd projects/ansible_dexter
9641 cd ~/projects/ansible_dexter
9675 cd projects/ansible_dexter
9684 cd ~/projects/sendgrid-test
9696 source /home/jim/projects/sns_python/venv/bin/activate
9707 source /home/jim/projects/sns_python/venv/bin/activate
9713 cd ~/projects/
9863 cp -r ~/projects/sendgrid-test/lib64 .
9865 cp -r ~/projects/sendgrid-test/share .
9962 cd projects
9972 cd projects/terraform-provider-aws
9979 cd projects/terraform-provider-aws
10003 cd projects/terraform-provider-aws
For me, this is something I do so frequently that I wanted to shorten that command to the minimum number of characters. I settled upon `hg` for “history grep”, so I created an alias as a line in my `~/.zshrc` file: `alias hg=”history | grep”`
The `~/.zshrc` (or `~/.bashrc` if you use `bash` as your shell,) is a file that gets read every time you log in and sets up your preferred environment.
Perfect! Now, if I want to search history for commands that included a particular string, I can just type `hg projects` and I will get the above output.
Well, nearly perfect…
As it happens, hg is also a command used by the Mercurial Source Code Management system. This is not a big issue for me, as I’m a loyal Git user, but it presents an interesting dilemma:
How do I protect against command name collisions in my aliases?
/bin/ls instead of just lsjimoconnell_hg, though a comma is of course much shorter.
I found this tip via the excellent ‘/r/commandline’ subreddit.
I haven’t used this technique much at all yet, so let me know what you think!
(This post is meant not only to describe a little shell function, but to introduce you to a powerful, yet simple concept.)
When I’m working with a command line, I will frequently need to create a directory and then immediately change to that directory.
Let’s say I’m starting a new project. I will cd into my ~/projects directory and then create a new folder where I’ll be putting my files. After creating the new folder, I’ll generally need to `cd` to that folder.
cd ~/projects
mkdir foo
cd foo
My first thought was to write a little `alias` that combined the two operations, but aliases don’t really handle parameters well, so I wrote it as a shell function. (I use the amazing `zsh` as my terminal shell, but this approach works just the same in bash.)
In the below example, the “$1” refers to the argument passed. “$2”, “$3” and so on also work. (Be sure to quote them!)
mgo () {
mkdir "$1"
cd "$1"
}
You can paste that into your shell and it will work for the duration of your session, but of course the better way of making it available is to put it in your `~/.zshrc` or whatever your ~/.profile file is in your shell of choice. For example, in OS X, you have to first create the file:
Start up Terminal.
Type "cd ~/" to go to your home folder.
Type "touch .bash_profile" to create your new file.
Edit .bash_profile with your favorite editor (or you can just type "open -e .bash_profile" to open it in TextEdit.
To use it, you need to have that file be re-read, so you can either close and re-open your terminal, `source` the file you just edited, or just paste the function in to have it be active for the current session.
I realize that this is an insanely simple thing to worry over, but so much of the Linux/Unix philosophy is about making things just a bit more efficient and shell functions are a very approachable way to craft your own customizations for efficiency.
Another Example:
If you’re a python programmer or student, you are probably aware of venv, the virtual environment for Python that create lightweight Virtual Environment on a directory-by-directory basis. If you have, you know that each time you move to a project directory, you need to type `./venv/bin/activate` to set up the environment. A clever function I found a while back takes over the `cd` command to check for a virtual environment and activate it if it exists:
cd () {
builtin cd $1
if [[ -d ./venv/bin ]]
then
source ./venv/bin/activate
fi
if [[ -d ./bin ]]
then
source ./bin/activate
fi
}
Both of the above examples are simple, but useful. Please leave a comment below if you know of any other good examples, or can think of a good use case for a function.
The other day I was writing about a method for Proximity Presence Detection using a Raspberry Pi and the Bluetooth radio in your smartphone.
(Basically, when your phone (or other Bluetooth device) gets within range of the Raspberry Pi’s Bluetooth, the MAC Address of the phone is noted and a script sends some data to an MQTT server, so you can determine with some confidence that you are home. It’s quick, lightweight and doesn’t require any software to be installed on the phone. Once the script notes your presence, you can do things such as turn on lights, have your home automation system greet you or do pretty much anything that computers can do.)
While setting it up, I noticed something odd, though–turning off Bluetooth didn’t really turn off Bluetooth, at least on my iPhone.
Here’s what I expected to see. The number in bold, right after “confidence” is the value that shows that the radio is active within range. When it disappears from range, the number tapers down over a period of 90 seconds or so:
{"confidence":"100","name":"J’s iPhone","scan_duration_ms":"1394","timestamp":"Sun Feb 03 2019 09:39:04 GMT-0500 (EST)"}
{"confidence":"83","name":"J’s iPhone","scan_duration_ms":"5150","timestamp":"Sun Feb 03 2019 09:39:47 GMT-0500 (EST)"}
{"confidence":"66","name":"J’s iPhone","scan_duration_ms":"5153","timestamp":"Sun Feb 03 2019 09:39:55 GMT-0500 (EST)"}
{"confidence":"50","name":"J’s iPhone","scan_duration_ms":"5148","timestamp":"Sun Feb 03 2019 09:40:04 GMT-0500 (EST)"}
{"confidence":"33","name":"J’s iPhone","scan_duration_ms":"5148","timestamp":"Sun Feb 03 2019 09:40:12 GMT-0500 (EST)"}
{"confidence":"16","name":"J’s iPhone","scan_duration_ms":"5149","timestamp":"Sun Feb 03 2019 09:40:20 GMT-0500 (EST)"}
{"confidence":"0","name":"J’s iPhone","scan_duration_ms":"5149","timestamp":"Sun Feb 03 2019 09:40:28 GMT-0500 (EST)"}
To test it, I tap the home button on my phone and swipe up, revealing the Control Center:
Oddly, when I clicked the Bluetooth icon, it dimmed, as you’d expect, but it was still appearing as 100% in my script:
{"confidence":"100","name":"J’s iPhone","scan_duration_ms":"1218","timestamp":"Sun Feb 03 2019 10:00:26 GMT-0500 (EST)"}
{"confidence":"100","name":"J’s iPhone","scan_duration_ms":"385","timestamp":"Sun Feb 03 2019 10:00:57 GMT-0500 (EST)"}
{"confidence":"100","name":"J’s iPhone","scan_duration_ms":"1197","timestamp":"Sun Feb 03 2019 10:01:28 GMT-0500 (EST)"}
My first suspicion was that my installation of the script was at fault or that perhaps I was reading the wrong MAC address, but I checked those and quickly started to suspect my phone.
To test this, I shut off my phone and saw the confidence percentage drop gracefully to zero.
I turned the phone back on and the script performed perfectly, detecting the radio instantly.
(“Airplane Mode,” I found, did actually turn off the radio, as you would expect.)
Next, I connected a Bluetooth speaker and started playing music, while watching my script. Hitting the Control Center icon disconnected the speaker, but didn’t shut off the radio.
This struck me as odd behavior, but not surprising, given that it’s Apple and I have little confidence in them to be upfront about something like this.
I dug around a bit this morning and came across an Apple support article titled “Use Bluetooth and Wi-Fi in Control Center with iOS 11 and later.” In that rather vaguely titled article, it says the following:
In iOS 11 and later, when you toggle the Wi-Fi or Bluetooth buttons in Control Center, your device immediately disconnects from Wi-Fi and Bluetooth accessories. Both Wi-Fi and Bluetooth will continue to be available, so you can use these important features:
AirDrop
AirPlay
Apple Pencil
Apple Watch
Continuity features, like Handoff and Instant Hotspot
Instant Hotspot
Location Services
Not in the list, but according to another source, Apple’s AirPod earphones apparently also do not disconnect when Bluetooth is deactivated from Control Center. (I have no AirPods to test this.)
Also worthy of note is that if you do it this way, it will automatically restore Bluetooth to full capacity the next day at 5:00 AM local time. Why it does this isn’t clear, but it’s probably so your Bluetooth alarm clock will still function or something.
If you go into IOS’s “Settings” app, you the Bluetooth controls behave as you would expect–turning off Bluetooth actually does shut off the radio, so I doubt this is anything beyond Apple making their own proprietary technologies and accessories work better than third-party stuff.
Anyway, I thought it was interesting enough to dig into a bit and share with you all.
The big takeaway is that using Control Center is not the same as controlling things via the Settings app and may not behave as expected.
Threat Level: Mostly Harmless ;-)
Go and order online ed kamagra with discounted price from FaastPharmacy
You must be logged in to post a comment.