MongoDB Tuning: disable-transparent-hugepages fix for Debian/Ubuntu

MongoDB Tuning: disable-transparent-hugepages fix for Debian/Ubuntu

Seeing: “WARNING: /sys/kernel/mm/transparent_hugepage/defrag is ‘always’.” ?

Run the following commands to quickly do what MongoDB describes at greater length.

  # Currently just debian
  sudo curl -sSL -o /etc/init.d/disable-transparent-hugepages https://gist.githubusercontent.com/justsml/5e8f10892070072c4ffb/raw/disable-transparent-hugepages
  sudo chmod 755 /etc/init.d/disable-transparent-hugepages
  sudo update-rc.d disable-transparent-hugepages defaults

References:

  1. https://docs.mongodb.org/v3.0/tutorial/transparent-huge-pages/
When AI Fails & the Crashing Robotic Cars

Robotic Cars: More accidents (now With Less Injury & Damages!)

Google’s self-driving cars are apparently in 2x as many accidents as human drivers - If you think this is just buggy new tech, way too complex from the get-go - well, you’re partially right. An important detail I should share: as of Dec 2015 virtually all accidents were not the robots fault.

The accidents are caused by humans drivers unfamiliar with robotic drivers. Furthermore, Google has programmed the cars to obey the law in absolute terms - never speeding, difficulty merging in dense or fast freeway traffic. This opens up a bunch of legal, ethical questions -

  • Is it ever ok:
    • to avoid a rear-end collision if it means driving past the walkway at a red light assuming path is clear ?
    • to drive on the sidewalk to avoid a bicycle cutting across lanes? - is it ok for AI to accept this breach in ‘rules of the road’

I understand Google’s approach, especially when trying to minimize liability: always follow the rules - logically it follows that you cannot really be at fault if you always observe the law.

There would be massive liability if an accident happened because of intentionally designed ‘flexibility’ around the laws.

Don’t let the future escape us

The future will still arrive, even if the robots drive like octagenarians.

Perhaps a simple fix for now would be to use bright red flashing LEDs (think school buses) to warn human drivers they are about to rear-end an innocent robot.

I would be more comfortable with a car which had tiers of observance and rule adherance. To my mind this would be much closer to how humans drive.

Imagine 3-tiers of system-perception as follows: (decision & other layers omitted for simplicity)

  1. base: follows laws with annoying precision
  2. local: flexibile adjustments based on current traffic - to allow merge on the highway if say, 10MPH+ is needed. Conversely if the traffic is simply going too fast, the car should be smart enough to pull over to avoid being a nuissance to other drivers.
  3. 360: calculate ANY potential extreme collision risk/avoidance measures - driving on a shoulder or

This would likely require a smart balancing act - say tier 1 detects an immenent accident which cannot be avoided following the laws, it would then shift all processor power into tier 3 - hopefully finding a creative way to avoid harm.

Robotic cars are on the cusp of being technically smarter & faster than any human driver. Accept it. Welcome it.

References

  1. http://nn.cs.utexas.edu/pages/research/neat-warning/
  2. http://www.claimsjournal.com/magazines/idea-exchange/2014/09/29/255161.htm
Higher Order Programming

Array- and Set-based Coding Techniques

An Anti-Pattern?

This is an exploration of advantages gained when you code everything as an array. (Using Jedi concepts from SmallTalk)

Here’s some guiding principles:

  1. All input is array-like. Even if an array of 1.
  2. Higher level functions should generally accept AND return arrays. (Except for callback methods for loops: map/reduce/each/filter)
  3. 99 out of 100 devs code suffers from what I call acute schema surplusage syndrome.
  4. Beware bloated class-backed models - with all the predictable trappings: fragile instance state w/ so many levers and knobs to mess with, DB transactions, sql locks, async/mutexing (that always works first time), using idiomatic property getter/setters, and your public/private/final/etc usage is solid, right?

  5. So let me take a common problem and shoehorn ~add~ some set-based musings.
  6. A hypothetical Blog Site has lots of Articles, and has even more Posts (Comments).
  7. Let’s add a delete method (below) - but with support for both singular OR arrays.
package net.danlevy.why.java___why.you.got.all.the.dots____it.must.be.all.the.factories;

public class Post {
  public String   title;
  public Date     created;
  public String   message;

  public Post(String title, String message) {
    this.title    = title;
    this.message  = message;
    this.created  = new Date();
  }

  public Date isArchived() {
    return this.created < new Date(2015, 0, 1);
  }

  // Post.delete` can be called w/ a singular Post or an array of Post[]
  public static int delete(Post post) {
    Post<List> posts = new Post<List>(post);
    return delete(posts)
  }

  public static int delete(Post<List> posts) {
    return posts.map(DB.cleanupPost);
  }
}



Forgive me if my Java is a little rusty.

Beautiful Models (and Data)

Work-in-progress (updated Sep. 2015)

The issue we’ll examine is deceptively simple & subtle: Naming

I want to avoid the super-fancy-tech-lingo for this article; and hopefully I can illustrate the issue in a more useful fashion.

While covered in exhausting detail before, the subject matter often gets too technical for the novice programmer to draw any practical understanding. You probably don’t need to read this if the following makes sense: No-Sql denormalization strategy, or Boyce Codd Normal Forms

Recommended reading includes:

  1. Book: Code Complete
  2. http://phlonx.com/resources/nf3/
  3. https://en.wikipedia.org/wiki/Database_normalization

The Problem - by Example

Have you ever designed a data model (in code, Sql, or excel worksheets)? Does the following look familiar?

*** anti-pattern - don't copy-paste ***
* User
  - id
  - avatarUrl
  - email
  - passwordHash

* Agent
  - id
  - primaryPhoto
  - agentName
  - agentEmail
  - agentPhoneMain
  - agentEmailPrimary
  - agentPhonePrimary
  - agentAddressLine1
  - agentCompanyName
  - agentCompanyAddress
  - *userEmail* - 'Pointer' to User table ^^^

If this is familiar to you, I’ll bet you:

  1. Feel any change to your app will necessitate hours of arduous debugging.
  2. Fear ANY Changing Requirements schema refactor

The Cost of Bad (Naming) Habits

Let’s examine some of the subtle issues (probably familiar):

=======

Why is naming a field agentEmailPrimary the worst?

For starters, you are not creating an entirely new object unto the universe. Over-specificity has some traps:

  1. ‘Locked’ into highly specific name, means agentEmailPrimary probably make your views and related code 0% reusable, and featuring annoyingly recurring bugs like:
    • Data not syncing between tables (not obvious if user.email needs to propagate to agent.agentEmail or vice-versa - nevermind complexity of manually implementing where & how to enforce this ‘logic’ …)
    • Validation rules/logic are likely duplicated & inconsitent.
    • Increasingly, your project will resemble a shaky Jenga tower.
    • Fragility piles up with every single new file, as an extremely high attention to detail is required for even trivial changes

I know, you probably feel something like…

fuck this

A Solution

// Dan's Recommended Schema Consolidation:

User
  - id
  - role: ['agent', 'lead', 'admin']
  - name
  - phone
  - address
  - email
  - passwordHash
  - company
    - name
    - address

I removed the Agent table, as it didn’t contain fields which were uniquely related to Agents.

All changes were made with these general ideas in mind:

  1. Eliminate unessesary tables. If you have a few dozen tables, this step is mandatory.
  2. Try merge related tables. Important if you are coming from a SQL background to No-SQL
  3. Delete redundant data collection (e.g. remove ActivityLogs table if replaced by Google Analytics)
  4. Try keeping all field names to a single word/noun/pro-noun.
  5. There is no such thing as Agent.agentEmail or Agent.agentPhonePrimary. Period.
  6. By using Highly Specific Names, you cast-in-stone a specific level of code-reusability and durability, well, specifically ZERO %.
  7. Don’t think you are doing yourself any favors with crap like this User.profileSummaryEmail (where ‘profile’ could include contact details for a personal ads site) . This is probably a good point to create a new table, say Profiles which includes Profiles.email.

Work-in-progress (updated Sep. 2015)

AngularJS v2.0 and the Impending Schism

I think we are witnessing the Python 2->3 ‘Conversion’ all over again. AngularJS v2.0 introduces too many changes. Not least of which is TypeScript, which is a big ask amidst the finalization of JS’s latest version: ES6.

Let me say clearly: I love TypeScript. I seccretly wish the TC-39 meetings had produced it… They didn’t. However, They came up with another (totally different), also-awesome spec…

While TypeScript compiles to JavaScript, it doesn’t mean you blindly copy & paste ‘compiled’ TypeScript. It effectively becomes required learning, as to understand annotated AngularJS 2.0 TypeScript.

Now, newbies must climb ‘Mount TypeScript’ before they can even start assembling an Angular app (with some level of understanding).

I have a feeling how this might go…

endless loop

Oh well, I’ll add it to the Newbie training list: somewhere between Basic Shell Usage and Gulp or Grunt? Godsend+Misery!

Anyway, I hope this works out…

everything is going to be ok

Tag Cloud

Jekyll::Drops::SiteDrop