my-notes

Bonus - Databases

01. Databases in development process

Developers need database for local development.

• Option 1:
  • Each developer install mysql DB locally
  • Each developer has their own DB with own test data
  • 👍 can’t mess up with someonelse’s test DB data
  • 👎 start from empty DB (needs realistic test data)
  • 👎 have to install and setup DB for EVERY developer
• Option 2:
  • DB hosted remotely
  • 👍 no need for local installation (start coding right away)
  • 👍 test data available right from the start
  • 👎 can’t play around without affecting others
• Ideal solution: have both!

In your application code you configure the DB connectivity. You have to tell the library, WHICH DB to talk to and HOW to AUTHENTICATE with that DB.

DO NOT HARDCODE CREDENTIALS IN THE CODE

Define in 1 place as environment variables (or a .env file).

Depending on environment (dev, test, prod), connects to different DB. Each ahs different endpoints and credentials (PROD more secured).

Define a variable in code, and then set endpoint/credentials from outside for each environment.

Configure externally instead of hardcode.

• Java/Spring Application with Spring properties file.
  • Define the values in the properties file
  • Reference the value in the Java App
• Logging level
  • log more in development mode
  • log less in production mode (e.g.: only errors)
• As a DevOps Engineer, you need to know:
  • how to configure DB
  • how to setup DB
  • how to manage DB
    • replication
    • backups
    • restore DB

DataBase Types

• Why so many databases?
• How are they different?
• Which one should I use for my App?

Key Value Databases

• Popular implementations:
  • Redis
  • Memcached
  • etcd from Kubernetes
• Features:
  • unique key
  • no joins
  • data stored in memory:
    • 👍 very fast
    • 👎 limited storage
  • not used as primary DB
• best for:
  • caching
  • message queue

Wide Column Databases

row key	column \n name	column \n age
mike	mike	21

• Popular implementations:
  • Cassandra
  • Apache HBase
• features:
  • schema-less
  • queries similar to SQL
  • 👍 scalable
  • 👎 no joins
  • 👎 limited compared to relational DB
  • not used as primary DB
• best used for:
  • time series
  • IoT records
  • historical records

Document Databases

• Popular implementations
  • MongoDB
  • Dynamo DB
  • CouchDB

Document

"name": "Mike"
"age": 29

• features
  • schema-less
    • documents are grouped in Collections, and Collections can be organized in a hierarchy
    • not recommeded for “connected data”
  • no joins
  • 👎 slower writes (compared to relational DB)
  • 👍 faster to read
  • 👍 easy to get started
  • 👍 can be used as primary DB
  • recommended for situations where you still don’t know how the data is going to be structured
• best used for:
  • mobile apps
  • game apps
  • CMS
  • most apps

Relational Databases

• Popular implementations:
  • MySQL
  • PostgreSQL
  • Oracle DB
• features
  • most used!
  • structured data
  • schema and data types needs to be created first
  • data is organized in tables
  • normalizing to avoid duplicated data
  • 👍 ACID (Atomicity, Consistency, Isolation, Durability)
    • No half-changes are updated in database
  • 👎 difficult to scale
    • Note: CockroachDB was specifically designed to solve the scalability issue.

Graph Databases

Nodes and Edges

• Popular implementations:
  • Neo4j
  • Dgraph
• features
  • directly connect entities
  • edges are the relationships
  • 👍 easier to query
• best used for:
  • graphs
  • patterns
  • recommendation

Search Databases

• Popular implementations:
  • ElasticSearch
  • Solr
• Features:
  • Search Database through massive data entries
  • Full text search in efficient and fast way
  • Similar to document-oriented database
  • creates index of words

Wrap Up

• Choose database type based on your App needs
• Or Combinations
  • For most data: Relational
  • To handle fast search: Search Engine
  • Cache: Key Value