Install Mamba (python3) in Fedora39.

1. Download the bash script to install miniforge from conda-forge. Yes, now miniforge contains mamba and its the recomended way to do it. Detail instructions are located at: GitHub - conda-forge/miniforge: A conda-forge distribution.

   As noticed before, I have a folder named Build for downloading and building the software that I need.

   Do not forget to update Fedora before installing something.

   sudo dnf update
   

   mkdir ~/Build/mamba
   cd ~/Build/mamba
   wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
   bash Miniforge3-Linux-x86_64.sh
   

2. Follow the instructions instructions in the bash script, prompted in the screen. Because I am a single user, I kept the default settings. Adjust as needed. The installation’s location is: ~/miniforge3/

3. Use mamba instructions the same way as conda. Most of them work the same way, be aware of what you are doing. Here are some handy examples:

   Create an environment with Python 3.9 use:

   mamba create -n python39 python=3.9
   

   Do not forget to activate the environment before working on your projects, and installing the required libraries. Also do not forget to deactivate the environment after you are done.

   mamba activate python39
   

   mamba deactivate
   

   To install a single library, for example pandas, do:

   mamba install pandas
   

   To install a list of libraries saved in the file requirements.txt use:

   mamba install --file requirements.txt
   

   Here is the requirements.txt content:

   altair==4.1.0
   jupyterlab==3.2.6
   ipywidgets==7.6.5
   matplotlib==3.5.1
   pandas==1.3.5
   jupyterlab-dash==0.1.0a3
   seaborn==0.11.1
   python-dotenv==0.19.0
   psycopg2==2.9.3
   SQLAlchemy==1.4.25
   

Installation and first steps in Spyder-IDE

I grew up as a Data Scientist using RStudio. Without it, I would not have follow this career path. Since then I have been looking for an IDE that provides the same functionality that RStudio offers; in Python. This is where Spyder comes into play. I am not a fan of Jupyter Notebooks.

1. Use mamba to install Spyder. Follow the conda instructions at: Installation Guide; Spyder 5 documentation

   mamba create -c conda-forge -n spyder-env spyder numpy scipy pandas matplotlib sympy cython
   

   Notice the -c command to force the source at conda-forge, and not to i.e. Anaconda. At the same time other libraries were also installed.

2. Activate the spyder-env environment and install the required additional plugins. Line profiler, Notebook and Terminal. The first as its name suggests, aids in line profiling . The second allows the interaction with Jupyter Notebooks. And the last one integrates a terminal to the IDE.

   mamba activate spyder-env
   mamba install spyder-line-profiler spyder-notebook spyder-terminal -c conda-forge
   

3. Launch Spyder-IDE from the terminal; remember to activate the spyder-env .

   mamba activate spyder-env
   spyder
   

4. Here is the RStudio Window Layout for the Exploratory Data Analysis - Seattle, WA Project.

   And here is the updated version:)

   

5. Selecting the correct Kernell.

   By default Spyder uses the Python version in which it was installed. In the Spyder installation, I did not specified, it used the latest version installed in Fedora; 3.12.1.

   To select the correct kernel to run your project. Select Preferences from the Tools menu and go to the Python interpreter section, there chose the appropriate interpreter. For details see Sypder-IDE FAQ; using existing environment

6. To finish. Save your work. Close the Sypder-IDE window and in the terminal deactivate the environment.

   mamba deactivate
   

7. Final Notice

   This Spyder-IDE setup has a big problem running in Fedora. It shows the following Warning:

   ❯ spyder
   Warning: Ignoring XDG_SESSION_TYPE=wayland on Gnome. Use QT_QPA_PLATFORM=wayland to run on Wayland anyway.
   fromIccProfile: failed minimal tag size sanity
   

   The root of this problem is the display version that by default the Fedora39 Workstation standard version comes with. Since Fedora25 is Wayland instead of Xorg11. Sypder is setup by default to display its interface in Xorg. And, although the IDE is displayed, the produced graphs appear as a black box. This is an ongoing issue to be solve. The work around is to save the images as pdf or png files and open them in a separate program. This kills the benefits of having an IDE. So, be aware of this issue. I will update this post once I find a solution.

Additional information:

Welcome to Mamba’s documentation!

GitHub - conda-forge/miniforge: A conda-forge distribution.

Spyder - IDE for scientific programming in Python

Wayland vs. Xorg: Will Wayland Replace Xorg?

In this post I document how to setup Fedora 38 in WSL. The setup is based on information from Install WSL | Microsoft Learn and from Install Fedora 37 or earlier on Windows Subsystem for Linux (WSL) - DEV Community.

Installing Fedora 38 Container Based Build

1. Ensure that Virtual Maschine is enabled in the BIOS and in Windows. For the first step follow the computer manufacturer. Then follow the steps in Install WSL | Microsoft Learn.

2. Ensure WSL2 is running in Windows:

   wsl --set-default-version 2
   

   If errors arise, consult google or youtube to find how to setup WSL 2 in windows. Just search for How to setup WSL2. As an alternative you can read Install WSL | Microsoft Learn.

3. Download the latestest Fedora Container Base build from Koji Fedora eg. Fedora-Container-Base-38-20230619.0. Be sure that the State has a green check mark and not a yellow X or red -. As this mean that the build was either canlleded or failed. Download the Fedora-Container-Base-38-20230619.0.x86_64.tar.xz file and extract it, eg. WinRar. The content from the folder with the HEX name should containt the layer.tar file. This is the required file to install Fedora.

4. Create a folder where Fedora will be installed. eg: C:\Users\User\AppData\Local\WSL2\Fedora38

5. Import the Fedora distribution into WSL. In this example, I am naming it Fedora38

   wsl --import Fedora38 C:\Users\User\AppData\Local\WSL2 C:\Users\User\Downloads\layer.tar
   

6. Now review the installed distros:

   wsl -l -v
     NAME        STATE           VERSION
   * Ubuntu      Stopped         2
     Fedora38    Stopped         2
   

   In this case Ubuntu is also installed. the -v parameter checks for the version. Ensure it is 2. Remembered that it was setup like that in Step number 2.

7. Setup the Fedora38 as the default distribution:

   wsl -s Fedora38
   

8. To lunch it now just type

   wsl
   

9. If you want to run other installed distro, eg. Ubuntu run:

   wsl -d Ubuntu
   

   Review the list of installed distros to find the correct name, see point 6.

10. In widnows I use Oh My Posh with Windows Terminal, there you can open a Terminal with Fedora, Powershell, Ubuntu, etc.

11. If you want to find which other distros can be installed from terminal run:

    wsl --list --online
    

Aferter install setup

There are additional steps to completelly setup Fedora in wsl. These are not required from distros on the online list.

1. When running Fedora the first time, it will run as root. Be carefull on what you do.

2. Update the distro

   dnf update
   

   If you cannot update the distro, there are issues resolving the DNS. See the Ensure DNS is functioning in the post Install Fedora 37 or earlier on Windows Subsystem for Linux (WSL) - DEV Community

3. Install basic system utilities

   dnf install util-linux
   

   Exit the distro

   exit
   

   and therminate it in PowerShell:

   wsl -t Fedora38
   

   restart Fedora38

4. It is not recomended to run root as default. Create a user with sudo previledges.

   dnf install passwd
   useradd -G wheel newuser
   passwd newuser
   

   Exit and reboot Fedora distro as in step 3.

5. Lunch Fedora38 with the newuser

   wsl -u newuser
   

6. If the seupt worked properly you should now see:

   [newuser@computername WindowsUser]$
   

   instead of

   [root@computername WindowsUser]$
   

   check for the sudo previledges:

   sudo cat /etc/shadow
   

   You should get the information regarding the users, it should include the newuser information.

7. Setup the user as default login.

   printf "\n[user]\ndefault = newuser\n" | sudo tee -a /etc/wsl.conf
   

   This code adds the new user to the wsl.conf file

8. Restart Fedora38 as in step 3. And you are done.

Additional Steps

The Fedora Container Base build has just the basic information lets install neofetch to find the current information

sudo dnf install neofetch
neofetch


             .',;::::;,'.                newuser@WINDOWS-COMPUTER-NAME
        .';:cccccccccccc:;,.            ---------------------
      .;cccccccccccccccccccccc;.         OS: Fedora Linux 38 (Container     
    .:cccccccccccccccccccccccccc:.       Kernel: 5.15.90.1-microsoft-st   
  .;ccccccccccccc;.:dddl:.;ccccccc;.     Uptime: 8 mins
 .:ccccccccccccc;OWMKOOXMWd;ccccccc:.    Packages: 417 (rpm)
.:ccccccccccccc;KMMc;cc;xMMc:ccccccc:.   Shell: bash 5.2.15
,cccccccccccccc;MMM.;cc;;WW::cccccccc,   Terminal: Windows Terminal
:cccccccccccccc;MMM.;cccccccccccccccc:   CPU: Intel i7-5500U (4) @ 2.39
:ccccccc;oxOOOo;MMM0OOk.;cccccccccccc:   Memory: 302MiB / 3868MiB
cccccc:0MMKxdd:;MMMkddc.;cccccccccccc;
ccccc:XM0';cccc;MMM.;cccccccccccccccc'
ccccc;MMo;ccccc;MMW.;ccccccccccccccc;
ccccc;0MNc.ccc.xMMd:ccccccccccccccc;
cccccc;dNMWXXXWM0::cccccccccccccc:,
cccccccc;.:odl:.;cccccccccccccc:,.
:cccccccccccccccccccccccccccc:'.
.:cccccccccccccccccccccc:;,..
  '::cccccccccccccc::;,.

There a re only 417 installed packages.

Install fedora documentation.

grep -v nodocs /etc/dnf/dnf.conf | sudo tee /etc/dnf/dnf.conf
sudo dnf install -y man man-pages

These ensure that the manual pages are installed. It requires to reinstall all packages.

for pkg in $(dnf repoquery --installed --qf "%{name}"); do sudo dnf reinstall -qy $pkg; done

Clean the installation

sudo dnf clean all

ZSH

Follow the steps in Fedora 38 Setup - KDE - Server Version | An Intersection of Engineering, Agriculture, Data Science and Innovation to setup ZSH

The following packages and steps are required.

dnf install util-linux-user
chsh -s $(which zsh)

Save the distribution

We can backup the distribution. It allows to save the distro current state. This is achived by running in PowerShell the following commands:

mkdir BackupWSL
cd .\BackupWSL
wsl --export Fedora38 20230626_Fedora38_WSL.tar 

The command takes several minutes. Be patient.

Texlive and others

Fedora provides different levels of TeXLive. From bare minimal to full. These common packages are texlive-scheme-basic, texlive-scheme-medium and texlive-scheme-full. Also there are the contex, gust, minimal and textex. I am starting wih the basic and installing packages as needed.

sudo dnf install pandoc texlive-scheme-basic texstudio 

I used LuaLaTeX to create my resume and to write letters. The letters have an specific structure, eg. my and recipent information, topic, date and its content.

The templates require additional libraries and fonts. These are the following:

sudo dnf install texlive-tcolorbox texlive-fontawesome texlive-academicons texlive-silence texlive-sourcesanspro texlive-cormorantgaramond texlive-tex-gyre texlive-babel-german texlive-babel-spanish 

To automate letters creation, eg. Job applications, I save the dynamic information in a Markdown file, in yaml format. This file is passed then to LuaLaTeX with Pandoc aid.

pandoc metadata.md --template letter_template.tex --pdf-engine=lualatex -o output.pdf

The code creates a pdf file (output.pdf) based in a latex template (letter_template.tex) and a yaml structure in markdown file format (metadata.md).

PDF edition

Some times I need to add or remove pages to a PDf file. For this purpose I use pdftk-java. It is an implementation available in Fedora from the pdftk program written in C.

Here is a use case example. I want to create a new file (output.pdf) that will have the pages 6 to 15 from the bigfile_with15pages.pdf file. The code to extract these pages is the following:

pdftk bigfile_with15pages.pdf cat 6-end output output.pdf 

The comand uses the parameter cat to extract pages 6 to 15 from the bigfile_with15pages.pdf file. And it saves them in output.pdf

Other use case is the joining several pdf files. It is also achieved with the parameter cat. Here is an example:

pdftk resume.pdf certificates.pdf cat output resume_with_certificates.pdf 

The command joints the files resume.pdf and certificates.pdf. They are joined in order of apperance. The parameter output generates the resume_with_certificates.pdf.

PDF file size

Reduce a pdf its size. it is playing with the image resolution. In the following example is setup for an image of 180-x 180 psi. With a jpeg compression of 40%.

convert -density 180x180 -quality 60 -compress jpeg base.pdf output_base.pdf 

So, I am trying GNOME. After installing the Workstation ISO and setting up as in the previous post, the rebooting problem remains. It seems that is a recurring problem with the nouveau drivers (ABRT Analytics - Report #617657 - xorg-x11-drv-nouveau in gf100_gr_reset). This means that NVIDIA drivers are required; at least is what I presume. While using Ubuntu and loading the proprietary NVIDIA drivers, the workstation worked without any issues. In this post I am documenting how to install them.

The post image was created with midjourney. Prompt: /imagine Create an abstract paint that illustrates the fusion of engineering and innovation with Data Science, navigating the crossroads of expertise to create sustainable and efficient solutions using oil as the medium, with the impasto technique, emphasizing depth and contrast, and featuring a stunning midnight blue color palette. –no Letters, Text, Signature, Watermark, Logo, Stamp, Branding, Trademark, Copyright, Registered Trademark, Human, Person, People. –aspect 7:4 –s 750 –v 5 –q 2

Fixing Logitech drivers

Before starting with configuring with the video card drivers, I setup the Logitech mouse.

sudo dnf update
sudo dnf install solaar 

Install NVIDIA Drivers

I followed the how to documentation in Howto/NVIDIA - RPM Fusion. The documentation provides all the required steps before and after the drivers configuration. Read the complete page before installing the drivers.

Configure RPM Fusion

sudo dnf update
sudo dnf install https://mirrors.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm https://mirrors.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm
sudo dnf groupupdate core

Secure Boot

1. Install akmods package.

   sudo dnf install akmods
   

2. Read the README file to review the proper steps to follow.

   nano /usr/share/doc/akmods/README.secureboot
   

3. Enroll process with Akmods

   sudo /usr/sbin/kmodgenca
   

4. Enter the certificate information

   -----
   You are about to be asked to enter information that will be incorporated
   into your certificate request.
   What you are about to enter is what is called a Distinguished Name or a DN.
   There are quite a few fields but you can leave some blank
   For some fields there will be a default value,
   If you enter '.', the field will be left blank.
   -----
   Organization Name (eg, company) [akmods local]: erickcantu
   Organizational Unit Name (eg, section) [akmods]:ecantu
   Email Address [akmods@localhost.localdomain]:erick@erickcantu
   Locality Name (eg, city) [None]:
   State or Province Name (full name) [None]:
   Country Name (2 letter code) [XX]:
   Common Name (eg, your name or your server's hostname) [akmods local signing CA]:
   

5. Enroll new keypair with certificate and enter its password. It will be asked when the computer reboots.

   sudo mokutil --import /etc/pki/akmods/certs/public_key.der
   input password: 
   input password again: 
   

6. Rebooting the system is needed for MOK to enroll the new public key. BE SURE YOU HAVE A WIRED KEYBOARD!!!

   On next boot MOK Management is launched and you have to choose “Enroll MOK”.

   Choose “Continue” to enroll the key or “View key 0” to show the keys already enrolled.

   Confirm enrollment by selecting “Yes”.

   You will be invited to enter the password generated above. WARNING: keyboard is mapped to QWERTY!

   The new key is enrolled, and system ask you to reboot.

   You can confirm the enrollment of the new keypair once the system rebooted with:

   mokutil --list-enrolled | grep Issuer
   

   or with:

   sudo mokutil --test-key /etc/pki/akmods/certs/public_key.der
   

   which should return

   /etc/pki/akmods/certs/public_key.der is already enrolled
   

House keeping

I already have 10 enrolled keys, which I have been creating in the last days. Lets remove the ones that are not needed. For details see: uefi - Is it possible to delete an enrolled key using mokutil without the original .der file? - Ask Ubuntu

1. Create a mokkeys directory.

   mkdir mokkeys
   cd mokkeys
   

2. List the enrolled keys by exporting them to the mokkeys directory. I use zsh, therefore the lsa command. Use ll in bash.

   mokutil --export
   lsa
   

3. Review the keys content. Be careful to select only the keys that are not needed. You do not want to re-install everything again.

   mokutil -l | less
   

4. Delete they selected keys by erasing the files name MOK000x.der. The command is going to ask you for a password that is needed after rebooting. Use the same password for each key. Ideally the same one that was used to enroll them.

   sudo mokutil --delete MOK000x.der
   

5. You need to reboot the system. You will come to the blue screen that is going to ask you if you want to delete the selected keys. You do this part in a bash. Not per individual key. Reboot the system.

Install NVIDA Drivers from RPM Fusion

sudo dnf update -y
sudo dnf install akmod-nvidia

Installed:
  akmod-nvidia-3:530.41.03-1.fc38.x86_64                                                                  
  egl-gbm-1.1.0-4.fc38.x86_64                                                                             
  egl-wayland-1.1.11-3.fc38.x86_64                                                                        
  nvidia-settings-3:530.41.03-1.fc38.x86_64                                                               
  xorg-x11-drv-nvidia-3:530.41.03-1.fc38.x86_64                                                           
  xorg-x11-drv-nvidia-cuda-libs-3:530.41.03-1.fc38.x86_64                                                 
  xorg-x11-drv-nvidia-kmodsrc-3:530.41.03-1.fc38.x86_64                                                   
  xorg-x11-drv-nvidia-libs-3:530.41.03-1.fc38.x86_64                                                      
  xorg-x11-drv-nvidia-power-3:530.41.03-1.fc38.x86_64      

The documentation provides a suggestion to install cuda. But, I will try with Howto/CUDA - RPM Fusion in the next post.

Reboot the computer.

IT BOOT TO A BLACK SCREEN

It happened again, the same bug as with the KDE version. The boot was to a black screen. This bug is well documented, and according to RPM Fusion documentation, the solution is in place. However, for the 530.41.03 driver version is not. You need to run the commands that are part of the KMS (Kernel Mode Setting”). Now I wonder if I should go back to the minimalist KDE server version.

sudo grubby --update-kernel=ALL --args='nvidia-drm.modeset=1'

Ensure the suspend command works

sudo dnf install xorg-x11-drv-nvidia-power
sudo systemctl enable nvidia-{suspend,resume,hibernate}

The package is already installed. I presume the configuration is also available. Will see if there are any issues after it suspends.

Video Acceleration

Video reproduction or streaming requires video acceleration. No need for additional ffmpeg drivers or media drivers. The following code is enough.

sudo dnf install nvidia-vaapi-driver libva-utils vdpauinfo

Nouveau verification

Review that Nouveau is not running the following code should produce nothing.

lsmod |grep nouveau

Change monitor scale options

By default in Fedora you can scale by 100%. This means that you can select 100% or 200% scale in the display menu. It is not possible to select something in between, e.g. 150%. This can be modified by running:

gsettings set org.gnome.mutter experimental-features "['scale-monitor-framebuffer']"

See: How to Enable Fractional Scaling on Fedora Linux - OMG! Linux

In the next posts, I will review installing LaTeX, modifying GNOME, maybe trying Hyland, but definitely trying EWW Widgets. Also we will review CUDA; this is going to be intersting.

For some time I tried with Linux Mint and popOS in a laptop. It was fine, still gnome in a laptop. But, since the beginning of the year I have been using Fedora 37 on an old Surface Go2, it runs amazing; for a Surface Go2 with 3Gb of RAM. No complains, 12 Gb of space to run in comparison to the 30 somethings of Windows. It came back to life, yes the camera still does not work as intended, but everything else is great. Thanks to the people in GitHub - linux-surface/linux-surface: Linux Kernel for Surface Devices. It is an amazing work.

With all these in mind and because I decided, “Finally”, to move to Linux as my main driver and have Windows as backup. And, since Fedora 38 just lunched…. well here we are. This is my personal implementation. It is mainly to document the steps that I followed. And by no means is the best implementation. I should have stayed with the Fedora 38 Workstation; Nvidia seems not to like it (in another Post I will look into the NVIDIA drivers journey), but its gnome….

The post image was created with midjourney. Prompt: /imagine Create an abstract paint that illustrates the fusion of engineering and innovation with Data Science, navigating the crossroads of expertise to create sustainable and efficient solutions using oil as the medium, with the impasto technique, emphasizing depth and contrast, and featuring a stunning midnight blue and dark green color palette. –no Letters, Text, Signature, Watermark, Logo, Stamp, Branding, Trademark, Copyright, Registered Trademark, Human, Person, People. –aspect 7:4 –s 750 –v 5 –q 2

Install Fedora Server with KDE

• Download the server ISO and lunch it in GitHub - ventoy/Ventoy: A new bootable USB solution.

• In the anaconda installer:

  • Create a user

  • Select to install from mirror list use: Fedora Mirrors - 38

  • Select Software to Install. Use KDE for this Version. Unselect firefox and libreoffice.

  • Select the disk to use. Use Automate version.

Initial Configuration

Because I am installing the server version and adding KDE without firefox and libreoffice, the default size for / is 15Gb. I expanded it to the maximum possible size. First update the installation and install git, kitty and zsh.

[erick@fedora ~]$ sudo dnf update -y
[erick@fedora ~]$ sudo dnf install git kitty zsh

Now expand root size. Fore details see: Fedora Server Install not Using entire disk. - YouTube

I am not convinced this is the correct way to move forward with the implementation. I have the feeling that home is out of the scope in this implementation. And, that I am not using root as intended. I do not have a message board, but feel free to contact me if you have any ideas or opinions about this.

[erick@fedora ~]$ sudo lvdisplay
[sudo] password for erick:
  --- Logical volume ---
  LV Path                /dev/fedora/root

[erick@fedora ~]$ sudo lvresize -r -l +100%FREE /dev/fedora/root
   Logical volume fedora/root successfully resized.

The first part, lvdisplay, provides the required path name for lvresize.

Now create a folder to build the setup. Eventually everything that needs to be build lands here.

mkdir Build
cd Build
sh -c "$(curl -fsSL https://raw.github.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"

git clone https://github.com/denysdovhan/spaceship-prompt.git "$ZSH_CUSTOM/themes/spaceship-prompt" --depth=1
ln -s "$ZSH_CUSTOM/themes/spaceship-prompt/spaceship.zsh-theme" "$ZSH_CUSTOM/themes/spaceship.zsh-theme"

nano ~/.zshrc

once in nano, configure oh-my-zsh prompt inside .zshrc. Replace the ZSH_THEME=XXX with:

ZSH_THEME="fino"

Finally add a couple of pluggins to highlight the commands and generate auto suggestions.

git clone https://github.com/zsh-users/zsh-syntax-highlighting.git ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-syntax-highlighting

git clone https://github.com/zsh-users/zsh-autosuggestions ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-autosuggestions

nano ~/.zshrc

Configure the plugins inside .zshrc

change plugins=(git) to plugins=(git zsh-autosuggestions zsh-syntax-highlighting)

Compile .zshrc

source ~/.zshrc

Install Vivaldi with dnfdragora package installer.

mkdir vivaldi
cd vivaldi
wget https://downloads.vivaldi.com/stable/vivaldi-stable-6.0.2979.18-1.x86_64.rpm

After downloading it. Go to the folder location within Dolphin. Select run with dnfdragora package installer by doing right click to the rpm file. The 6.0.2979.18-1.x86_64 version is the latest at the time this post is being publish. Review the latest version.

Install the following programs and applications

• Install AppImageLauncher follow the steps in Home · TheAssassin/AppImageLauncher Wiki · GitHub. Use dnfdragora package installer. Same procedure as with Vivaldi

• Onlyoffice AppImage, use its AppImage

• Marktext AppImage latest release

• neofetch

sudo dnf install neofetch

Setup GitHub

Create SSH key and setup. Follow the instructions at Connecting to GitHub with SSH - GitHub Docs.

Setup Visual Studio Code

Follow the instructions in: Running Visual Studio Code on Linux

sudo rpm --import https://packages.microsoft.com/keys/microsoft.asc
sudo sh -c 'echo -e "[code]\nname=Visual Studio Code\nbaseurl=https://packages.microsoft.com/yumrepos/vscode\nenabled=1\ngpgcheck=1\ngpgkey=https://packages.microsoft.com/keys/microsoft.asc" > /etc/yum.repos.d/vscode.repo'

dnf check-update
sudo dnf install code

Install and configure pyenv inside .zshrc

git clone https://github.com/pyenv/pyenv.git ~/.pyenv
echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.zshrc
echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.zshrc
echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n eval "$(pyenv init --path)"\nfi' >> ~/.zshrc

Install Jekyll and setup for Basically Basic Jekyll Theme

Follow the installation steps Jekyll on Linux

In brief install prerequisites.

sudo dnf install ruby ruby-devel openssl-devel redhat-rpm-config @development-tools

Add gems to path with:

nano ~/.zshrc

# Install Ruby Gems to ~/gems
export GEM_HOME="$(ruby -e 'puts Gem.user_dir')"
export PATH="$GEM_HOME/bin:$PATH"
export GEM_BINN="$HOME/bin"
export PATH="$GEM_BINN:$PATH"

source ~/.zshrc

Install Jekyll and Bundler gems

gem update
gem install jekyll bundler

There was missing a library make: g++: No such file or directory.

Solve it installing

dnf install gcc-c++

Follow the instructions Basically Basic Jekyll Theme, run

bundle install

Ruby suggest to update it to its latest version. However a permission error was trigger.

A solution is to install it as a super user. However ruby shows the message: Do not run bundle install with root privileges. Because the Ruby gems was installed with dnf, it was installed as superuser. The right way to update ruby gems in this case is to use Fedora dnf. But this will not have the latest version. Which may not be needed in this case.

A solution would be to install Ruby Gems according to Download RubyGems / your community gem host. But that is out of the scope of my usage.

So far is were we are. I am still missing Pandoc, LuaLaTex, pdftk, TexLive and TexStudio. Advance data science setup, yes NVIDIA drivers…

With this purpose in mind, we can use a GPT solution to simplify our work. I am presenting a prompt structure. This structure is being used in ChatGPT-4 to create 3 possible prompts for Midjourney.

I) Your role
Take the role of a Prompt Engineer with bast experience in Midjourney. 

II) The Request Context:
You will use the following structure to create prompts:
Content, Medium, Style, Lighting, Color Palette, Composition.

Unless the variables values are given to you, generate random values based in the following definitions:
Medium: insert artistic medium here.
Style: insert an Artist or Genre. 
Lighting: insert a lighting type utilized in photography or painting.
Color Palette: insert the color pallet to use, verify that matches the style type.
Composition: insert cameras models, specific lenses, photography or image types, positional elements.

III) Your work:
I will provide your with content ideas. 
With these ideas you will help to write 3 different prompts to create images and art. 
Write in natural language, utilize specific, precise, and articulated language. 
Include the following text at the end of each promt:
"--no Human, People, Person, Letters, Numbers, Text, Signature, Watermark, Logo, Stamp, Branding, Trademark, Copyright, Registered Trademark.

Example - EDA King County Washington

For the EDA post, I added a new image header to the webpage. To create it, I used the following prompt.

An aerial recreation of King County Seattle Washington housing. Consider key sightseeing in Seattle. To be use in a realtor marketing campaign. The target market is high income. Medium: photo realistic image. Style: 4K photograph Lighting: emphasize depth and contrast Color Palette: early golden hour At the end of the prompt add --aspect 7:4

ChatGPT-4 provided the following prompts:

1. Design a captivating aerial image of King County Seattle Washington housing, incorporating significant Seattle landmarks, intended for a realtor marketing campaign, in a photo-realistic 4K photograph with lighting that accentuates depth and contrast, and showcasing a rich midnight blue color palette. –no Human, People, Person, Letters, Numbers, Text, Signature, Watermark, Logo, Stamp, Branding, Trademark, Copyright, Registered Trademark. –aspect 7:4

2. An aerial recreation of King County Seattle Washington housing. Consider key sightseeing in Seattle. To be use in a realtor marketing campaign. The target market is high income. Medium: photo realistic image Style: 4K photograph Lighting: emphasize depth and contrast Color Palette: early golden hour At the end of the prompt add –aspect 7:4

3. Compose an impressive aerial representation of King County Seattle Washington housing, highlighting key attractions in Seattle, suitable for a realtor marketing campaign targeting high-income clients, as a photorealistic 4K photograph with evocative lighting that highlights depth and contrast, and featuring an early golden hour color palette. –no Human, People, Person, Letters, Numbers, Text, Signature, Watermark, Logo, Stamp, Branding, Trademark, Copyright, Registered Trademark. –aspect 7:4

The provided prompt to Midjourney was:

Compose an impressive aerial representation of King County Seattle Washington housing, highlighting key attractions in Seattle, suitable for a realtor marketing campaign targeting high-income clients, as a photorealistic 4K photograph with evocative lighting that highlights depth and contrast, and featuring an early golden hour color palette. --no Human, People, Person, Letters, Numbers, Text, Signature, Watermark, Logo, Stamp, Branding, Trademark, Copyright, Registered Trademark. --aspect 7:4 --s 750 --v 5 --q 2

And the results are:

Prompts used in this website

Kickstarter

Design a vector illustration for a website header with the topic of Kickstarter, depicting a rocket launching to symbolize project success. The style should be modern and sleek, with dramatic lighting and a strong light source coming from behind the rocket. The color palette should include contrasting shades of blues, greens, and whites to convey the excitement of innovation. Compose the image with a diagonal layout, and have the rocket take up most of the frame for a dynamic effect. --no Human, People, Person, Letters, Numbers, Text, Signature, Watermark, Logo, Stamp, Branding, Trademark, Copyright, Registered Trademark. --aspect 7:3 --s 750 --v 5 --q 2

Result:

The Green City Solutions Group

Take a realistic 4K photograph of a 9-building complex focused on forecasting building energy demand, which includes a fast-food restaurant, standalone retail, strip mall retail, and 7 multi-family buildings. Visibly integrate solar panels, wind turbines, and heat pumps. Use warm, golden-hour sunlight for lighting to emphasize the eco-friendly nature of the buildings. The color palette should consist of earth tones, with subtle green and blue accents to symbolize energy efficiency. Compose the image with an elevated perspective, highlighting the buildings and energy sources from an interesting angle. --aspect 7:3 --no Human, People, Person, Letters, Numbers, Text, Signature, Watermark, Logo, Stamp, Branding, Trademark, Copyright, Registered Trademark. --s 750 --v 5 --q 2

Image Prompts for Midjourney

Create a composite landscape photograph for the header of a personal webpage, combining the mountains of Mexico, including Cañon del Sumidero, Cañón Namurachi, Cañón de Aquetzalli, and Barranca del Cobre, with the Black Forest in Germany, adding mist to create a surreal and mystical atmosphere, using a color palette of dark greens and blues, and composing the image with strong foreground elements to provide depth and scale. No text. No signature. --v 5 --s 750

This image still had a Signature in the third variation. Which I thought the No Signature part at the end would have been enough. But it did not work. It lead me to better understand the negation process in Midjourney. After this image and other similar ones, I added the command --no Human, People, Person, Letters, Numbers, Text, Signature, Watermark, Logo, Stamp, Branding, Trademark, Copyright, Registered Trademark. instructions.

Introduction

Since energy prices are continuing to rise and the future of the energy situation is rather uncertain, cities may want to investigate the energy consumption of different building sectors to predict future energy demand and identify areas where energy can be saved.

Energy demand forecasting is fundamental for an energy utility’s decision making on:

• Grid stability

• Planning power supply activities

• Reducing energy wastage

Since the data available consists of a series of energy consumption values taken sequentially with a fixed time interval over four years, time series analysis and models are ideal for this problem.

Methodology

Time series data was analyzed for trend and seasonality.

It seemed suitable to forecast the energy demand for 24 hours, as weather predictions get less accurate further in the future. Else, the power suppliers energy management is mainly focused on a 24 hour period.

Different models were applied and compared:

• Baseline (last years values)
• Linear Regression
• Polynomial Regression
• SARIMAX
• Prophet
• TBats
• XGBoost
• Random Forest

Main Results

A small trend in the net energy demand over 4 years was discovered with a slight increase over the first 3 years and a decrease in the 4th year (corresponding to the trend in the weather data). A clear yearly seasonality is found with the highest energy demand in summer (due to air conditioning) and the lowest energy demand in winter (due to mild winters). Furthermore a weekly as well as a daily seasonality was identified.

Conclusion

Tree-based machine learning models (Random forest and XGBoost) produced forecasts with the lowest mean absolute error compared to the observed data.

Links

The complete project is available at the The Green City Solutions Group: Forecasting building energy demand through time series analysis and machine learning GitHub repository. At the end of the analysis a presentation was done to the Neuefische team and other trainees. The presentation slides are available here, while the presentation can be watch at: The energy of tomorrow - YouTube.

1. Identifying patterns and relationships: EDA helps us understand how different variables within a dataset relate to each other. This is crucial for identifying potential predictor variables and building accurate predictive models.

2. Detecting outliers and anomalies: EDA can help us identify any unusual or unexpected data points, which can be critical for avoiding misleading results and improving the accuracy of our models.

3. Understanding the distribution of the data: By visualizing the distribution of the data, we can gain a better understanding of the central tendency and spread of the data, which can inform the choice of statistical methods and models.

4. Improving the quality of the data: EDA can help us identify issues with the data such as missing values, duplicates, or inconsistent formats. By fixing these issues, we can improve the quality of the data and increase the accuracy of our results.

5. Providing insights for data cleaning and pre-processing: EDA helps us identify issues with the data that require cleaning and pre-processing, such as transforming variables to a more suitable form, or removing outlier or irrelevant data points.

In conclusion, EDA is an essential step in the data science process that provides valuable insights and improves the quality of the data. It helps us understand the structure of the data, identify potential predictor variables, and avoid misleading results. By taking the time to perform EDA, we can ensure that our data is well understood, and that our models are based on accurate and relevant data.

“The information in the relevance of EDA was generated on 20230208 using OpenAI’s ChatGPT (GPT-3.5) language model. While every effort has been made to ensure its accuracy, the information should not be considered a primary source.”

Utilizing EDA to De-tangle Seattle, US Real State Market

At the Neuefische bootcamp, the EDA work was performed to the house sales in the King County dataset. The dataset is publicly available at Kaggle. To simulate a world case scenario. Different stakeholders with individual requirements were previous established by the Neuefische team (Asigment). In this analysis I simulated a realtor company who conducted the EDA to fulfill the needs of a fictional investor named Timothy Stevens. Mr. Stevens sells houses and is interested in putting his investments in the market. He owns expensive houses in the county center, which are the analysis target. The best timing for him is within a year. He is open for renovation if profits rise with it.

The analysis

At Red Cedar Property Advisors (RCPA) we specialize in providing the best advice to buy or sell your real state properties. We cover the King County in Washington State.

Our customer Timothy Stevens asked for our advice. He is interested in selling some of his properties in the center of the County within the next year. A key question that he brought to us was: Is it worth it to renovate the properties? Although he is opened in doing it, it will only be done if there is a profit benefit.

Main results:

The mean prices per zip code in the King County are quite disperse. The price range goes between 2.4 and 11 million. Because of this disperse, through the analysis I identified that anything bigger than 4 million is assumed its an outlier.

Unfortunately, our customer the properties in the county’s center are not reflected in these prices. Note to the reader: In the assessment description, I did not had information on the exact location of the stakeholder properties. Because of that, I use the city of Seattle as the base for the analysis

Is it worth to remodel?

Quite understandable the property and construction size correlate with the price. The bigger the property and its construction, the higher the market price. In our analysis we discover other factors that also have a slight correlation with the price in the center of the county . Current house condition and house grade (~0.63).

The house condition relates to the maintenance stage of the house, e.g. if it requires repair or how much work on it is required. While house grade relates on how good its construction meets the building code. From not meeting it, to a custom design.

In our analysis we found that improving the house condition or grade do not had an impact in the price. This result is only valid for the center of the city of Seattle and do not represent the whole County. Therefore RCPA gave the recommendation: do not remodel. Sell as is.

Takeaways

• The minimum price to sell is $ 675,000.

• House Condition and Grade drive the price.

• Do not remodel, it is not profitable in the city of Seattle.

Links

The analysis is available at the EDA_Project_KC_Housesales GitHub repository. At the end of the analysis a presentation was done to the Neuefische team and other trainees. This presentation is available at: EDA_Project_KC_Housesales Presentation.instuctions

Machine Learning to Predict Kickstarter Success

As part of the Neuefische data science bootcamp, we learn different Machine Learning algorithms, methods and models. We applied them in our second project. Which I conducted it with my colleague Sue Leen Wong. In our Machine Leaning project we analyzed data to predict the success of launching a crowdfunding project in Kickstarter.

The Project

Introduction

Kickstarter, founded in 2009, is a crowdfunding platform where project creators can raise money from the public, circumventing traditional avenues of investment. It has an all-or-nothing funding model, whereby a project is only funded if it meets its goal amount; otherwise no funds are collected.

A huge variety of factors contribute to the success or failure of a project on Kickstarter. Some of these factors are able to be quantified or categorized, which allows for the construction of a model to attempt to predict whether a project will succeed or not.

The goal of this project is to predict if a Kickstart project will succeed or fail through using Exploratory Data Analysis and supervised Machine Learning models.

More generally, the aim is to help potential project creators as well as potential investors assess what their chances of success on Kickstarter will be.

The Stakeholder

Mr. Jacoby Stevens

• Retired entrepreneur

• Likes supporting crowdfunding projects

• Is looking for the next big idea

• Would like us to predict for him the likelihood of success or failure

• Wants to support the projects most likely to succeed

Baseline Model

• Projects based in USA are likely to be successful

• Evaluation Metric: We train the model based on precision. Precision is the ratio of correctly predicted positives out of all of the results that were predicted positive.

• Score for baseline model: Precision = 57.8%, Accuracy = 55.2%

Takeaways and Future Work

• Improvement in accuracy from 55.2% to 72.0% over the baseline model with XGBoost.

• Best project categories to invest are:

  • Comics, dance or publishing categories

  • Projects featured as “staff pick” when launched

  • Projects from Hong Kong, Japan or Great Britain

• Worst project categories to invest are:

  • Technology

  • Food

Links

The analysis is available at the Predicting Kickstarter Project Success with Machine Learning GitHub repository. At the end of the analysis a presentation was done to the Neuefische team and other trainees. This presentation is available at: ML to predict Kickstarter success.