mongodb: mongo shell and insert data

July 26, 2017

The post discusses using mongoDB shell to insert data.

software version

bash-3.2$ mongo
MongoDB shell version v3.4.6

connect to mongodb database

bash-3.2$ mongo
MongoDB shell version v3.4.6
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.4.6
Server has startup warnings: 
2017-07-26T20:43:13.356+0800 I CONTROL  [initandlisten] 
2017-07-26T20:43:13.357+0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2017-07-26T20:43:13.357+0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2017-07-26T20:43:13.357+0800 I CONTROL  [initandlisten] 
2017-07-26T20:43:13.357+0800 I CONTROL  [initandlisten] 
2017-07-26T20:43:13.357+0800 I CONTROL  [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000

database, collection, and document

  • A mongoDB server has many databases.
  • A mongoDB database has many collections.
  • A mongoDB collection has many documents.
  • A mongoDB document is a JSON format data.

insert a document into test database’s member collection

  • The default database is test.
  • By default, the mongoDB server has two databases which have data.
  • The example inserts a document {‘name’: ‘chengyihe’} into member collection of test database.
  • WriteResult({ “nInserted” : 1 }) indicates that the insertion operation is success.
  • After inserting data into test database, “show dbs” also shows test database.
> db.getName()
test
> show dbs
admin  0.000GB
local  0.000GB
> db.member.insert({'name': 'chengyihe'})
WriteResult({ "nInserted" : 1 })
> show dbs
admin  0.000GB
local  0.000GB
test   0.000GB

list data in database

  • db.getName(): show current database name
  • show collections: list a collection in current database
  • db.member.find(): find all documents in member collection
> db.getName()
tes
> show collections
member
> db.member.find()
{ "_id" : ObjectId("5978953fe2861159432c64ed"), "name" : "chengyihe" }

conclusion
The post shows how to insert data into mongoDB database.

Advertisements

mongodb: mongod and mongo shell

July 26, 2017

The post discusses mongod and mongo shell.

software version

bash-3.2$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.11.6
BuildVersion:   15G1510
bash-3.2$ mongod --version
db version v3.4.6
git version: c55eb86ef46ee7aede3b1e2a5d184a7df4bfb5b5
OpenSSL version: OpenSSL 1.0.2l  25 May 2017
allocator: system
modules: none
build environment:
    distarch: x86_64
    target_arch: x86_64
bash-3.2$ mongo --version
MongoDB shell version v3.4.6
git version: c55eb86ef46ee7aede3b1e2a5d184a7df4bfb5b5
OpenSSL version: OpenSSL 1.0.2l  25 May 2017
allocator: system
modules: none
build environment:
    distarch: x86_64
    target_arch: x86_64

what is mongod
A server side mongoDB executable.

what is mongo shell
A client side mongoDB interactive shell.

run mongod as the the mongoDB server in localhost at port 27017
By default, the mongod runs the server at port 27017.

bash-3.2$ mongod -dbpath=$(pwd)/db
2017-07-26T20:43:12.449+0800 I CONTROL  [initandlisten] MongoDB starting : pid=51719 port=27017 dbpath=/Users/chengyihe/workspace/mongodb/db 64-bit host=nanshouoshisMBP
2017-07-26T20:43:12.450+0800 I CONTROL  [initandlisten] db version v3.4.6
2017-07-26T20:43:12.450+0800 I CONTROL  [initandlisten] git version: c55eb86ef46ee7aede3b1e2a5d184a7df4bfb5b5
2017-07-26T20:43:12.450+0800 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2l  25 May 2017
2017-07-26T20:43:12.450+0800 I CONTROL  [initandlisten] allocator: system
2017-07-26T20:43:12.450+0800 I CONTROL  [initandlisten] modules: none
2017-07-26T20:43:12.450+0800 I CONTROL  [initandlisten] build environment:
2017-07-26T20:43:12.450+0800 I CONTROL  [initandlisten]     distarch: x86_64
2017-07-26T20:43:12.450+0800 I CONTROL  [initandlisten]     target_arch: x86_64
2017-07-26T20:43:12.451+0800 I CONTROL  [initandlisten] options: { storage: { dbPath: "/Users/chengyihe/workspace/mongodb/db" } }
2017-07-26T20:43:12.453+0800 I -        [initandlisten] Detected data files in /Users/chengyihe/workspace/mongodb/db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2017-07-26T20:43:12.454+0800 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=3584M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2017-07-26T20:43:13.356+0800 I CONTROL  [initandlisten] 
2017-07-26T20:43:13.357+0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2017-07-26T20:43:13.357+0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2017-07-26T20:43:13.357+0800 I CONTROL  [initandlisten] 
2017-07-26T20:43:13.357+0800 I CONTROL  [initandlisten] 
2017-07-26T20:43:13.357+0800 I CONTROL  [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
2017-07-26T20:43:13.531+0800 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/Users/chengyihe/workspace/mongodb/db/diagnostic.data'
2017-07-26T20:43:13.533+0800 I NETWORK  [thread1] waiting for connections on port 27017

run mongo shell as mongoDB client
By default, mongo shell connects to mongoDB at 127.0.0.1:27017.

bash-3.2$ mongo
MongoDB shell version v3.4.6
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.4.6
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
        http://docs.mongodb.org/
Questions? Try the support group
        http://groups.google.com/group/mongodb-user
Server has startup warnings: 
2017-07-26T20:43:13.356+0800 I CONTROL  [initandlisten] 
2017-07-26T20:43:13.357+0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2017-07-26T20:43:13.357+0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2017-07-26T20:43:13.357+0800 I CONTROL  [initandlisten] 
2017-07-26T20:43:13.357+0800 I CONTROL  [initandlisten] 
2017-07-26T20:43:13.357+0800 I CONTROL  [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
>

List databases in mongoDB
Use “show dbs” to list all databases.

> show dbs
admin  0.000GB
local  0.000GB

conclusion
The post discusses how to use mongoDB shell to connects and send commands to mongoDB server.

mongodb: install mongodb in Mac OS X

July 26, 2017

The post discusses installing mongoDB in Mac OS X.

software version

bash-3.2$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.11.6
BuildVersion:   15G1510

what is mongodb
A non-sql database.

install mongodb

bash-3.2$ brew update
bash-3.2$ brew install mongodb
Updating Homebrew...
==> Installing dependencies for mongodb: openssl
==> Installing mongodb dependency: openssl
==> Downloading https://homebrew.bintray.com/bottles/openssl-1.0.2l.el_capitan.bottle.tar.gz
######################################################################## 100.0%
==> Pouring openssl-1.0.2l.el_capitan.bottle.tar.gz
==> Using the sandbox
==> Caveats
A CA file has been bootstrapped using certificates from the SystemRoots
keychain. To add additional certificates (e.g. the certificates added in
the System keychain), place .pem files in
  /usr/local/etc/openssl/certs

and run
  /usr/local/opt/openssl/bin/c_rehash

This formula is keg-only, which means it was not symlinked into /usr/local,
because Apple has deprecated use of OpenSSL in favor of its own TLS and crypto libraries.

If you need to have this software first in your PATH run:
  echo 'export PATH="/usr/local/opt/openssl/bin:$PATH"' >> ~/.bash_profile

For compilers to find this software you may need to set:
    LDFLAGS:  -L/usr/local/opt/openssl/lib
    CPPFLAGS: -I/usr/local/opt/openssl/include

==> Summary
�  /usr/local/Cellar/openssl/1.0.2l: 1,709 files, 12.1MB
==> Installing mongodb
==> Downloading https://homebrew.bintray.com/bottles/mongodb-3.4.6.el_capitan.bottle.tar.gz
######################################################################## 100.0%
==> Pouring mongodb-3.4.6.el_capitan.bottle.tar.gz
==> Caveats
To have launchd start mongodb now and restart at login:
  brew services start mongodb
Or, if you don't want/need a background service you can just run:
  mongod --config /usr/local/etc/mongod.conf
==> Summary
�  /usr/local/Cellar/mongodb/3.4.6: 18 files, 267.5MB

Run mongdbb in localhost
By default, mongod uses /data/db as database directory. If another place if preferred, then -dbpath flag could help assign another directory as mongoDB directory.

bash-3.2$ mkdir db
bash-3.2$ mongod -dbpath=$(pwd)/db
2017-07-26T20:11:49.965+0800 I CONTROL  [initandlisten] MongoDB starting : pid=51583 port=27017 dbpath=/Users/chengyihe/workspace/mongodb/db 64-bit host=nanshouoshisMBP
2017-07-26T20:11:49.965+0800 I CONTROL  [initandlisten] db version v3.4.6
2017-07-26T20:11:49.965+0800 I CONTROL  [initandlisten] git version: c55eb86ef46ee7aede3b1e2a5d184a7df4bfb5b5
2017-07-26T20:11:49.965+0800 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2l  25 May 2017
2017-07-26T20:11:49.965+0800 I CONTROL  [initandlisten] allocator: system
2017-07-26T20:11:49.965+0800 I CONTROL  [initandlisten] modules: none
2017-07-26T20:11:49.965+0800 I CONTROL  [initandlisten] build environment:
2017-07-26T20:11:49.965+0800 I CONTROL  [initandlisten]     distarch: x86_64
2017-07-26T20:11:49.965+0800 I CONTROL  [initandlisten]     target_arch: x86_64
2017-07-26T20:11:49.965+0800 I CONTROL  [initandlisten] options: { storage: { dbPath: "/Users/chengyihe/workspace/mongodb/db" } }
2017-07-26T20:11:49.966+0800 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=3584M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2017-07-26T20:11:50.437+0800 I CONTROL  [initandlisten] 
2017-07-26T20:11:50.437+0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2017-07-26T20:11:50.437+0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2017-07-26T20:11:50.437+0800 I CONTROL  [initandlisten] 
2017-07-26T20:11:50.437+0800 I CONTROL  [initandlisten] 
2017-07-26T20:11:50.437+0800 I CONTROL  [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
2017-07-26T20:11:50.500+0800 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/Users/chengyihe/workspace/mongodb/db/diagnostic.data'
2017-07-26T20:11:50.706+0800 I INDEX    [initandlisten] build index on: admin.system.version properties: { v: 2, key: { version: 1 }, name: "incompatible_with_version_32", ns: "admin.system.version" }
2017-07-26T20:11:50.706+0800 I INDEX    [initandlisten]          building index using bulk method; build may temporarily use up to 500 megabytes of RAM
2017-07-26T20:11:50.712+0800 I INDEX    [initandlisten] build index done.  scanned 0 total records. 0 secs
2017-07-26T20:11:50.713+0800 I COMMAND  [initandlisten] setting featureCompatibilityVersion to 3.4
2017-07-26T20:11:50.713+0800 I NETWORK  [thread1] waiting for connections on port 27017

conclusion
The post shows how to install mongoDB in Mac OS X and the result.

nodejs: url module and query

July 24, 2017

The post discusses url module and query.

version

bash-3.2$ node -v
v8.1.4

sample source code

  • url.parse() transforms a url string into a url object
  • url.href transforms a url object into a url string
  • url.parse(urlString).query returns the query string of the url
  • url.parse(urlString, true).query returns the query object of the url
const querystring = require('querystring')
const url = require('url')

const urlString = 'https://user:pass@sub.host.com:8080/p/a/t/h?query=string#hash'
console.log(urlString)

const urlObj = url.parse(urlString)
console.log(urlObj)

const urlString2 = urlObj.href
console.log(urlString2)

const urlObj2 = url.parse(urlString2)
console.log(urlObj2)

console.log(url.parse(urlString).query)

console.log(url.parse(urlString, true).query)

sample output

bash-3.2$ node app.js 
https://user:pass@sub.host.com:8080/p/a/t/h?query=string#hash
Url {
  protocol: 'https:',
  slashes: true,
  auth: 'user:pass',
  host: 'sub.host.com:8080',
  port: '8080',
  hostname: 'sub.host.com',
  hash: '#hash',
  search: '?query=string',
  query: 'query=string',
  pathname: '/p/a/t/h',
  path: '/p/a/t/h?query=string',
  href: 'https://user:pass@sub.host.com:8080/p/a/t/h?query=string#hash' }
https://user:pass@sub.host.com:8080/p/a/t/h?query=string#hash
Url {
  protocol: 'https:',
  slashes: true,
  auth: 'user:pass',
  host: 'sub.host.com:8080',
  port: '8080',
  hostname: 'sub.host.com',
  hash: '#hash',
  search: '?query=string',
  query: 'query=string',
  pathname: '/p/a/t/h',
  path: '/p/a/t/h?query=string',
  href: 'https://user:pass@sub.host.com:8080/p/a/t/h?query=string#hash' }
query=string
{ query: 'string' }

conclusion
The post demonstrates a sample code to understand url object.

nodejs: querystring and json

July 24, 2017

The post discusses querystring and json.

software version

bash-3.2$ node -v
v8.1.4

sample source code
Both query and JSON could be objects and strings.

  • qeurystring.parse() could transform a query string into a query object
  • querystring.stringify() could transform a query object into a query string
  • JSON.parse() could transform a json string into a json object
  • JSON.stringify() could transform a json object into a json string
  • name=chengyihe&id=32 is a query string
  • {“name”:”chengyihe”,”id”:”32″} is a json string
const querystring = require('querystring')

var queryObj = {name: 'chengyihe', id: '32'}
console.log(queryObj)

var queryString = querystring.stringify(queryObj)
console.log(queryString)

var queryObj2 = querystring.parse(queryString)
console.log(queryObj2)

var jsonObj = {'name': 'chengyihe', 'id': '32'}
console.log(jsonObj)

var jsonString = JSON.stringify(jsonObj)
console.log(jsonString)

var jsonObj2 = JSON.parse(jsonString)
console.log(jsonObj2)

sample output

bash-3.2$ node app.js 
{ name: 'chengyihe', id: '32' }
name=chengyihe&id=32
{ name: 'chengyihe', id: '32' }
{ name: 'chengyihe', id: '32' }
{"name":"chengyihe","id":"32"}
{ name: 'chengyihe', id: '32' }

conclusion
The post elaborate the difference between query and json.

nodejs: parse http get request data

July 24, 2017

The post discusses how to parse http get request data.

software version

bash-3.2$ node -v
v8.1.4

server source code

var http = require('http');
var url = require('url');

http.createServer(function (req, res) {
    res.writeHead(200, {'Content-Type': 'text/html'});
    var data = url.parse(req.url, true).query;
    console.log('req.url: ' + req.url);
    console.log('name: ' + data.name + ', id = ' + data.id);
    res.write('name: ' + data.name + ', id = ' + data.id);
    res.end('\n');
}).listen(3000);

output of running server

bash-3.2$ node app.js
req.url: /?name=chengyihe&id=32
name: chengyihe, id = 32

output of running client
By default, curl sends a http get request to a http server.

bash-3.2$ curl -i "http://127.0.0.1:3000/?name=chengyihe&id=32"
HTTP/1.1 200 OK
Content-Type: text/html
Date: Mon, 24 Jul 2017 14:21:53 GMT
Connection: keep-alive
Transfer-Encoding: chunked

name: chengyihe, id = 32

conclusion
The http.IncomingMessage.url has the http get data. This sample use require(‘url’).parse() to parse the data.

nodejs: parse http post data

July 24, 2017

This post discusses how to parse http post data in nodejs.

software version
$ node -v
v8.1.4

server source code

var http = require('http');
const querystring = require('querystring');

http.createServer(function (req, res) {
    res.writeHead(200, {'Content-Type': 'text/html'});
    if (req.method == 'POST') {
        var body = '';
        req.on('data', function(data) {
            body += data;
        });
        req.on('end', function(data) {
            console.log('body: ' + body);
            var parsedData = querystring.parse(body);
            res.write('name: ' + parsedData.name + '\n');
            res.write('id: ' + parsedData.id + '\n');
            res.end('process post finish\n');
        });
    }
}).listen(3000);

output of running client
curl sends post data with -d flag.

$ curl -i -d 'name=chengyihe&id=32' 127.0.0.1:3000
HTTP/1.1 200 OK
Content-Type: text/html
Date: Mon, 24 Jul 2017 14:05:56 GMT
Connection: keep-alive
Transfer-Encoding: chunked

name: chengyihe
id: 32
process post finish

output of running server

$ node app.js
body: name=chengyihe&id=32

conclusion
If http.IncomingMessage.method == ‘POST’, then this example use qeuerystring.parse() to parse post data on ‘end’ event.

nodejs: how to load json file

July 21, 2017

This post discusses how to load json files.

software version

bash-3.2$ node --version
v8.1.4

source code
The require function could help load json file as well as module. By default, the json file is put in node_modules directory.

const { name, id } = require('data');
console.log('name = ' + name)
console.log('id = ' + id);

json file

{
    "name": "chengyihe",
    "id": "32"
}

output

bash-3.2$ node app.js 
name = chengyihe
id = 32

conclusion
This post demonstrate how to load json files. It includes nodejs source code and the running output.

nodejs: how to create and load customised module

July 21, 2017

This post demonstrate an example to create and use customised module.

software version

bash-3.2$ node --version
v8.1.4

create an application
Create a simple application which is also a module.

console.log(module)

This application dumps its own module data within which module.paths indicates where it could load modules. It appears that modules are loaded from node_modules directory. The position of node_modules directory is searched from current directory, then form parent directory, then from parent’s parent directory, and so on.

$ pwd
/Users/chengyihe/workspace/node/myapp
bash-3.2$ node app.js
Module {
  id: '.',
  exports: {},
  parent: null,
  filename: '/Users/chengyihe/workspace/node/myapp/app.js',
  loaded: false,
  children: [],
  paths:
   [ '/Users/chengyihe/workspace/node/myapp/node_modules',
     '/Users/chengyihe/workspace/node/node_modules',
     '/Users/chengyihe/workspace/node_modules',
     '/Users/chengyihe/node_modules',
     '/Users/node_modules',
     '/node_modules' ] }

create node_modules directory to include module

bash-3.2$ mkdir node_modules

add hello module in node_modules directory

exports.name = 'chengyihe'
exports.city = 'taipei'

modify application to load hello module
Let the application load hello module, dump application module, and dump what hello module exports.

const hello = require('hello')
console.log(module)
console.log(hello);

output
The output of the application shows that it has a child module loaded from hello.js. What hello module exports is at line #24.

bash-3.2$ node app.js 
Module {
  id: '.',
  exports: {},
  parent: null,
  filename: '/Users/chengyihe/workspace/node/myapp/app.js',
  loaded: false,
  children: 
   [ Module {
       id: '/Users/chengyihe/workspace/node/myapp/node_modules/hello.js',
       exports: [Object],
       parent: [Circular],
       filename: '/Users/chengyihe/workspace/node/myapp/node_modules/hello.js',
       loaded: true,
       children: [],
       paths: [Array] } ],
  paths: 
   [ '/Users/chengyihe/workspace/node/myapp/node_modules',
     '/Users/chengyihe/workspace/node/node_modules',
     '/Users/chengyihe/workspace/node_modules',
     '/Users/chengyihe/node_modules',
     '/Users/node_modules',
     '/node_modules' ] }
{ name: 'chengyihe', city: 'taipei' }

conclusion
An application is a module. By default, it could load module from node_modules directory. The syntax to load module is require(‘module_name’).

nodejs: expressjs: use express generator to create web applications

July 20, 2017

This post discusses using express generator to create web applications.

software version

bash-3.2$ node -v
v8.1.4
bash-3.2$ npm -v
5.0.3
bash-3.2$ express --version
4.15.0

what is express generator
A command line tool to automatically generate web application skeleton.

install npm generator

bash-3.2$ npm install express-generator -g
/usr/local/bin/express -> /usr/local/lib/node_modules/express-generator/bin/express-cli.js
+ express-generator@4.15.0
added 7 packages in 1.979s
bash-3.2$ express -h

  Usage: express [options] [dir]

  Options:

    -h, --help           output usage information
        --version        output the version number
    -e, --ejs            add ejs engine support
        --pug            add pug engine support
        --hbs            add handlebars engine support
    -H, --hogan          add hogan.js engine support
    -v, --view   add view  support (dust|ejs|hbs|hjs|jade|pug|twig|vash) (defaults to jade)
    -c, --css    add stylesheet  support (less|stylus|compass|sass) (defaults to plain css)
        --git            add .gitignore
    -f, --force          force on non-empty directory

use express generator to create web applications

bash-3.2$ express --view=pug myapp

   create : myapp
   create : myapp/package.json
   create : myapp/app.js
   create : myapp/public
   create : myapp/routes
   create : myapp/routes/index.js
   create : myapp/routes/users.js
   create : myapp/views
   create : myapp/views/index.pug
   create : myapp/views/layout.pug
   create : myapp/views/error.pug
   create : myapp/bin
   create : myapp/bin/www
   create : myapp/public/javascripts
   create : myapp/public/images
   create : myapp/public/stylesheets
   create : myapp/public/stylesheets/style.css

   install dependencies:
     $ cd myapp && npm install

   run the app:
     $ DEBUG=myapp:* npm start

run the app: a web server

bash-3.2$ cd myapp && npm install
bash-3.2$ DEBUG=myapp:* npm start

> myapp@0.0.0 start /Users/chengyihe/workspace/node/myapp
> node ./bin/www

  myapp:server Listening on port 3000 +0ms

use browser to see the web page

what does npm start do
By default, it calls scripts.start property. In this case, scripts.start = “node ./bin/www”. Therefore, executing “npm start” is equivalent to executing “node ./bin/www”.

{
  "name": "myapp",
  "version": "0.0.0",
  "private": true,
  "scripts": {
    "start": "node ./bin/www"
  },
  "dependencies": {
    "body-parser": "~1.17.1",
    "cookie-parser": "~1.4.3",
    "debug": "~2.6.3",
    "express": "~4.15.2",
    "morgan": "~1.8.1",
    "pug": "~2.0.0-beta11",
    "serve-favicon": "~2.4.2"
  }
}

conclusion
The post shows how to use express generator tool to create a simple web server.


%d bloggers like this: