pavlokhmel.com

Benchmark GPU Blender classroom on Linux in command line

Measure: time to render 1 frame from Blender demo files "Classroom". OS: Rocky Linux 9.4 Blender release: 4.0.1 Download Classroom demo...

SLURM administrator commands

This is a collection of Slurm commands that are often in use. create account, add default account for user sacctmgr create account...

Nvidia GPU set power limit in Linux

Enable persistence mode on boot: systemctl enable nvidia-persistenced systemctl start nvidia-persistenced Or enable only for the current...

Benchmark GPU - PyTorch, ResNet50

ResNet50 is an image classification model. The benchmark number is the training speed of ResNet50 on the ImageNet dataset. Training...

Lustre - replace OST

I'm removing 4 OSTs. I want to replace it with a new disk and use the same index number. # lfs osts OBDS: 0: cluster-OST0000_UUID ACTIVE...

Cgroups v2 - limit memory and CPU usage for all users on Linux

Starting with systemd v239 it is more easy to set cgroups limits for all users on a system. This is an example of setting a memory limit...

Benchmark OSU Micro-Benchmarks

The OSU Micro-Benchmarks are a suite of benchmarks for measuring and evaluating the performance of MPI operations. I will use one of...

Benchmark LAMMPS

LAMMPS is a classical molecular dynamics code with a focus on materials modeling. I'm using a module from EasyBuild for compilers. Or...

Benchmark IOR

"ior" and "mdtest" parallel I/O benchmarks. They are coming together. Here is the focus on the IOR benchmark. Compile with GCC and...

Open OnDemand 3.0 install on Rocky Linux 8.7

This article is a collection of important notes with examples in addition to Open OnDemand documentation. There are many changes between...

PROXMOX (Debian 10, KVM) enabling SR-IOV for Mellanox Infiniband cards

There are 2 differences in configuration on: AMD or Intel based systems. ConnectX-3 or ConnectX-6 Mellanox cards. Update - I tested...

Lustre client compiled with Mellanox InfinibBand driver

CentOS 8.3 minimal, Mellanox drivers 5.1, Lustre 2.12.6 yum -y install tar perl kernel-rpm-macros lsof gcc...

Terraform - create storage_container in Azure for backups with azcopy on Linux

File with Azure environment variables setenv.sh #!/bin/sh export ARM_SUBSCRIPTION_ID="1113bbd1-cccc-aaaa-7777-e7254290dccc" export...

HPL benchmark on CentOS 8 with Intel compiler 2020

HPL benchmark solves a randomly generated dense linear system of equations in double floating-point precision (IEEE 64-bit) arithmetic...

Intel Optimized LINPACK Benchmark for Linux on CentOS 8

Download Intel Parallel Studio XE 2020 In this example Intel Parallel Studio XE 2020 Cluster Edition tar xf...

STREAM memory benchmark on CentOS 8 with GCC and Intel compiler

The STREAM benchmark is a simple synthetic benchmark program that measures sustainable memory bandwidth (in MB/s) and the corresponding...

HPC Challenge Benchmark compile CentOS 8

Follow the steps below to compile and run HPC Challenge Benchmark on CentOS 8. Download and unarchive source code: yum...

Trunk on port-channel between Mellanox and Dell switches with VLAN 1

Default vlan 1 does not behave as vlan 1 on mellanox. Best practice - do not use VLAN 1. But sometimes you need to use VLAN 1. Example...

Highly Available mount point with Pacemaker via iSCSI storage device on CentOS 7

1. Configure iSCSI target server and initiator clients 1.1. Servers Install 3 servers with following IP addresses. Storage server will...

Build SLURM rpms on CentOS 8

"rpmbuild -tb" is not enough on CentOS 8.1 to build SLURM 18.x and 19.x rpms. yum install epel-release yum --enablerepo=PowerTools...

Performance Co-Pilot (PCP) on CentOS 7

Starting with CentOS 7.4 package pcp-zeroconf is available. It will install and enable pmcd and pmlogger. yum -y install pcp-zeroconf...

Rebuild rpm mutter - Gnome 3 freezes after 50 days

Still there is no update for mutter package to fix this issue. Solution Example on CentOS 7.5 minimal: yum -y group install "Development...

OpenLDAP with SSL/TLS

This post shows 2 options: Self Signed Certificate without CA Self Signed Certificate with CA (Certificate Authority) Also it shows how...

Install AWX on CentOS 7

AWX is the upstream project from which the Red Hat Ansible Tower offering is ultimately derived. Read more: AWX project FAQ Install...

Rocks 6.2 - Could not find kernel image

This issue happens on some hardware nodes. After successful compute node install. First boot is hanging with message: TFTP prefix:...

Gitlab docker container - backup and restore

Two examples: backup and restore GitLab docker container with volumes backup and restore only useful data Gitlab is running in Docker...

Terraform example for OpenStack and Ansible

This example shows how to automate: Creation of two instances sequentially with Terraform on OpenStack cluster. Provisioning new...

Consul - create rpm package

RPM package is convenient way to automate consul installation on many RHEL or CentOS servers. But consul rpm is not provided by...

OpenStack (Newton) - issues after install

I followed this documentation: https://docs.openstack.org/newton/install-guide-rdo/ Below are mix of issue and how to fix them after...

OpenStack - convert an instance to an image

You cannot create image from a volume in-use. Volume status must be "available" to create image. So: 1. stop instance 2. create...

OpenStack (Newton) enable SSL

This post shows how to switch Horizon to HTTPS. Before start you should have working OpenStack Horizon through HTTP. My setup: OS:...

Python - yield - simple explanation

Meaning of this post is to create first simple understanding of the yield. Yield is one of the topics that maybe hard to understand in...

Kubernetes - troubleshooting and commands

This post is related to the previous one: Kubernetes – simple install on CentOS 7 with Web UI How to reset kubernetes configuration and...

Kubernetes - simple install on CentOS 7 with Web UI

This posts shows working kubernetes setup on CentOS 7.2.1511. Some steps are not covered in other articles on the internet. Current...

Docker - backup and restore container

Example below shows how to backup on server A and restore docker containers on server B. As example I'll use container created in the...

Docker - wordpress all-in-one container on CentOS 7

This example shows LAMP + Wordpress all in one docker container. This setup is not going to scale. And all-in-one makes it easy to...

Jenkins - simple master, slave setup and project

What we'll do: Install Slave Install Master Adding new slave node on jenkins master Create new project: get files from GitHub archive...

Docker containers with MySQL master-slave replication

This post shows process of moving MediaWiki to multiple docker containers setup with MySQL master-slave replication. I'll skip MediaWiki...

Download and Upload over an unstable connection in Linux

RSYNC has option --partial (keep partially transferred files) or -P same as --partial --progress. Script below uploads file over ssh and...

Puppet - quick start

Overview Very quick start on CentOS 7.x. Three servers: - master (puppet master) - web (puppet client, httpd and php will be installed)...

Create internal repository server on CentOS 7

This post shows how to create copy of CentOS 7 repositories: base centosplus extras updates epel And configure internal repository...

CentOS 7 PXE boot server and automated installation with kickstart

Install packages: yum -y install httpd xinetd syslinux tftp-server dhcp Copy files from iso image: mount -o loop...

OpenStack quick install on CentOS 7

This post shows only commands and file modifications without explanation. It is based on...

Docker - CentOS 7 container on Ubuntu

Create base image from current CentOS 7 system with one command: tar --numeric-owner --exclude=/proc --exclude=/sys --exclude=/boot...

Vagrant - backup and restore environment

This post is related to previous post "Vagrant with multiple machines and ports". 1. Backup boxes Remove file 70-persistent-net.rules on...

Ansible - deploy java application and Zabbix

This post is related to previous post "Vagrant with multiple machines and ports". We configure application and monitoring servers. And...

Vagrant with multiple machines and ports

Vagrant environment: 1. Prepare host system 1.1. Install Unubtu 14.04 x64 on host system. Download page:...

Hadoop setup on CentOS 7

Hadoop setup 3 servers with CentOS 7 minimal installation 64 bit. 192.168.0.100 master.example.com 192.168.0.101 slave01.example.com...

Jetty 9 and Hello World servlet

Download and unpack jetty wget http://ftp.heanet.ie/pub/eclipse/jetty/stable-9/dist/jetty-distribution-9.3.3.v20150827.tar.gz tar xzf...

LUSTRE - move data and remove OST and OSS

Check health status on OSS # cat /proc/fs/lustre/health_check device lustre-OST0000 reported unhealthy device lustre-OST0001 reported...

Benchmark parallel filesystem - IOR and mdtest.

OS: CentOS 6.5 Compile MPICH. yum install gcc gcc-gfortran gcc-c++ mkdir /lustre/software cd /lustre/software/ wget...

Hybrid MPI and OpenMP example

Install compiler sudo yum install gcc gcc-gfortran gcc-c++ Compile and install MPICH wget...

Create user on Active Directory from Linux.

Manage Active Directory from Linux. Changes on Windows Server 2008 r2 Add new role - Active Directory Certificate Services in Windows...

Chrooted (jail) home via SSH, SFTP and FTP

OS: RHEL 6.x, CentOS 6.x Simultaneous chrooted home via SSH, SFTP and FTP SSHD changes in file /etc/ssh/sshd_config #Subsystem sftp...

Lustre file system - simple deploy

OS: CentOS 6.3 MGS server: lustre-mgs IP 192.168.0.1 OSS server: lustre-oss-01 IP 192.168.0.2 Storage: local drive on OSS server...

Rocks Cluster and LVM

LVM is not supported by Rocks. But it is possible with additional manual work OS: Rocks 6 Add to a file...

HPC LINPACK benchmark

Contents Intro HPL + Intel MKL + Intel MPI HPL + ATLAS + MPICH2 HPL + GotoBLAS2 + Open MPI 0. Intro HPL is a portable implementation of...

Python and MySQL

Python script executes SQL request and prints output Basic example #!/usr/bin/python import MySQLdb db_connect = MySQLdb.connect(host =...

rrdtool - graph data

rrdtool - store and graph data. This example graph memory usage. Source data: free -m total used free shared buffers cached Mem: 499 214...

Network testing with dd and netcat

Network testing without special tools like iperf, netperf and other. On 10 GigE NICs single dd + netcat are limited by one CPU core....

SSH with expect in BASH, Perl, Python

There are special modules for Perl/Python for SSH connection. For example: Net::SSH2::Expect Net::SSH::Expect In BASH can be used SSH...

Solaris system information

1. Solaris physical memory usage Script to get free/unused memory on Solaris can be found with command vmstat. Without options, vmstat...

FTP client in BASH, Perl, Python

To compare script language for administration tasks. - connect to FTP server in passive mode - delete file - send file in binary mode...

Chrooted SSH/SCP with public-key on Solaris 10

OpenSSH 4.9p1 and higher support user chrooting SSH and SFTP. Latest stable:...

Veritas Storage Foundation and High Availability Solutions (quick install)

OS: SUSE Linux Enterprise Server 10.2 (x86_64) SF_HS version: 5.0MP3 1. Preinstallation Network: On linux-001 eth0 10.10.10.1 eth1...

Date manipulation in BASH, Perl, Python

To compare script language for administration tasks. Using default installation on Linux (RHEL 5, 6), Solaris (10/9), Mac OS X (10.6.7)....

Hardware raid controllers on Linux

1. For "HP Smart Array 5i/6i Controller" $ lspci -m | grep RAID 02:02.0 "RAID bus controller" "Compaq Computer Corporation" "Smart Array...

NFS4 server and client

OS: RHEL6 Protocol: TCP Default port: 2049 By default NFS packages are installed. System A (server) IP: 10.10.10.1 $ cat /etc/exports...

NTP server and client

OS: RHEL6 Protpcol: UDP Default port: 123 NTP installed by default. System A (server), IP 10.10.10.1 Edit /etc/ntp.conf: # Hosts on...

Remote logging. Server and client with rsyslog

OS: RHEL 6 Protocol: tcp or/and udp Default port: 514 By default RHEL 6 use rsyslog. System A (server), IP 10.10.10.1 Uncomment lines in...

Simple iSCSI target (server) and initiator (client)

iSCSI (Internet Small Computer System Interface) - an Internet Protocol (IP)-based to provide data storage over network. Protocol: tcp...

RPM build with single file

OS: RHEL 6 There is script /usr/sbin/khmel.sh Simple way to build RPM file, that will install this scrip on another systems. Install yum...

Encrypted disk with password key-file (LUKS)

Encrypt disk: cryptsetup luksFormat /dev/sdb Create mapper device and file system: cryptsetup luksOpen /dev/sdb testdisk mkfs.ext3...

Auto lock and unlock user account in Linux

If user entered his password incorrectly several times his account can be locked. sudo faillog -u khmel Username Failures Maximum Latest...

MySQL notes

Backup and restore database, reset password...

Disk operations in Linux (SAN, VMware, Veritas)

1. Scan for new SAN or VMware disk echo "- - -" > /sys/class/scsi_host/host0/scan Replace host0 with actual value. You can find...

SSH Authentication with Public-Key

Crete Private and Public key on serverA.khmel.org user "phn": ssh-keygen -t rsa without phrase. Simple copy to serverB.phn.org.ua with...

Chrooted SFTP (RHEL 5)

On RHEL 5 with default installed openssh chroot will not allow to login via ssh. (Solution: run separate sshd on another port). To use...

NIC bonding on Linux

Example: [phn.org.ua ~]$ sudo cat /etc/modprobe.conf . . . alias bond0 bonding file /etc/sysconfig/network-scripts/ifcfg-bond0...

XEN CentOS as host and guest

Installation and configuration in command line without GUI. Install yum install xen virt-manager kernel-xen chkconfig xend on reboot...

Software RAID (mdadm) on Linux

OS: CentOS 5.4 mdadm — software RAID management tool Raid device names partitions in raid: /dev/md0, /dev/md1 disks in raid: /dev/md_d0,...

Mail server (CentOS, Postfix, Dovecot, SASL2, MySQL, PostfixAdmin)

Content: Preparation DNS MySQL Postfix with MySQL + SASL2 Dovecot Apache + PostfixAdmin Web Interface (Roundcube) Install Amavisd-new,...

OpenLDAP on CentOS 5.5

1. Names dn - Distinguished Names cn - Common Name dc - domain component ou - organizational unit uid - user id 2. Install OpenLDAP 2.1....

Tomcat 5.5 on FreeBSD 7.3

1. Update ports tree portsnap fetch extract update 2. Install Tomcat 5.5 If TZUPDATE option was chosen, 2 packets should be downloaded...

Special runlevels in Linux

Special runlevels "s", "emergency", "init=/bin/sh" and single user mode (runlevel 1) Runlevel "1" — will run /etc/rc.d/rc1.d Runlevel...

SELinux examples

1. Commands and files: getenforce, setenforce - get and set SELinux mode: Enforcing, Permissive, Disabled....

Iptables examples

1. Close all ports except 5130 and 21 IPTABLES='/sbin/iptables' $IPTABLES -A FORWARD -s 192.168.10.164 -p tcp -m tcp --dport 5190 -j...

Red Hat Cluster Suit (RHCS) on CentOS

OS: CentOS 5.4 as XEN guests Command line installation and configuration Disable SELinux abd Firewall: system-config-securitylevel-tui...

Tracking of root actions on Linux

To track root actions after: 1. # sudo -s 2. # sudo -i 3. # sudo su - 4. # sudo mc (with \^o) 5. And if some script “vi test.sh” was...

RHEL 6 and CentOS 5 after install

RHEL 6, installation: Citrix client, VPNC, Java, Virtual Box guest drivers. Create repository for Installation DVD [root@localhost ~]#...

Bash - HTML report generator with sending mail

Script below will generate HTML report. Report will be sent as HTML email. #!/bin/sh echo "To: phn@khmel.org" > /home/phn/`date...

Bash - date calculation

In log file /var/log/khmel.org.log can be message like: 16.10.2010 00:05:59 KHMEL.ORG site find AASSxxRR0111 connection To convert date...

Bash - auto SSH with expect and sudo commands

Example with auto creating user on several servers: ./sshauto.sh ip-list.txt 'hostname; sudo useradd -m -c "PHN user2" -G phn phnname2...

Bash - changing files (using functions with counter in loop)

This script add to HTML file strings from another file. And check if these strings already exist. #!/bin/bash AddNewString() {...

Bash - rewriting files on FTP server

#!/bin/bash HOST='phn.org.ua' USER='phn' PASSWD='PhnPassw0rd!' ftp -p -n -v $HOST << EOT ascii user $USER $PASSWD prompt cd www # Only...

Network settings and tools in Linux (Red Hat, CentOS, Fedora)

Contents Configuration files Configuration tools Network console tools Firewall files and tools Enable temporary telnet connection on...

File encription by GPG and OpenSSL

Without public key GPG # encrypt gpg -c khmel.tar.gz # decrypt gpg khmel.tar.gz.gpg OpenSSL # encrypt openssl des3 -salt -in...

LVM examples

First LVM commands to get overview on server: pvs # report information about physical volumes vgs # report information about volume...

Vsftpd anonymous access

Install: yum install vsftpd Change settings /etc/vsftpd/vsftpd.conf: anonymous_enable=YES local_enable=YES write_enable=YES...

VPN mpd on FreeBSD

Install mpd: pkg_add -r mpd # or cd /usr/ports/net/mpd make && make install If external IP: 123.123.123.123 VPN IP: 10.10.10.0...

Software RAID (gmirror) in FreeBSD

Step-by-step operations in command line to setup Soft RAID. Before setup: sysctl kern.geom.debugflags=16 To create mirroring on disk...

Port forwarding (3 solution for FreeBSD)

Solution #1: firewall rules: FwCMD="/sbin/ipfw" outip="123.123.123.123" httpserver="192.168.1.2" NetIn="192.168.1.0" NetMask="24"...

Openvpn, server FreeBSD, clients Ubuntu and Windows XP

Server Install cd /usr/ports/security/openvpn make install Add to /etc/rc.conf openvpn_enable="YES" Configuration file cd...

VPN server on Debian Linux (pptpd)

OS: Debian 3.1 r4 We need latest stable kernel source to compile modified kernel, now it is linux-2.6.19.1 cd /usr/src ftp...

Apache and OpenSSL (Debian, FreeBSD)

Secure data transferring via Web OS: Debian GNU/Linux 3.1 r0a "Sarge" FreeBSD 5.4-RC1-i386 Packages: OpenSSL apache+mod_ssl Create SSL...

Reset root password on Linux, FreeBSD, Solaris (x86)

Linux Reset root password on Linux (RHEL/CentOS). Grub or Lilo should be without password or you should know that password. In GRUB menu...

DNS (bind) example configuration with master and slave

/etc/bind/named.conf options { listen-on port 53 { 127.0.0.1; 192.168.2.1; server-ip; }; allow-recursion { 127.0.0.1; 192.168.2.0/24; };...

Email send and check in Linux comand line

Example: mail box me\@khmel.org, login: "me", password: "testpass". 1. Send e-mail # Connect to server on TCP 25 port $ telnet...

Simplest Postfix with Courier configuration

Postfix Main config file /etc/postfix/main.cf, with only 2 lines: mydestination = khmel.org home_mailbox = Maildir/ Create user test:...