Anyone succeeded to chainload any operating system on AWS EC2 HVM instances?
According to this post, it is not possible because it requires low-level access to hypervisor but there is not much information why. Any developer could comment on limitations with AWS EC2 PV instance?
What about HVM instances? I tried different tests but can get the network working *after chainloading the kernel*.
To summarize the tests:
1/ compiled latest version of iPXE (ipxe.lkrn) with an embedded script to boot from a network server
2/ created an HVM instance
3/ configured Grub to boot from ipxe.lkrn
4/ created an ipxe script on the network server to control chainloading
5/ reboot the EC2 instance
What is working:
- iPXE load and can find network (netfront)
- iPXE can connect to my network server and download various stuff (kernel, initrd, other iPXE scripts, etc.)
- imgtrust compiled and enforcing (chainloading fails as expected if downloaded content signature don't match)
- kernel loads and find initrd
- kernel can find disk device
- kernel *seems* to find a network device but not working
I tried CentOS 7.0, 7.1, 7.2 (and RHEL 7.2), CoreOS, RancherOS but none of them can bring up the network device.
If using DHCP:
- CentOS/RHEL distribution show "RTNETLINK answers: file exists" and fails back to the emergency shell
- CoreOS show nothing
If using Static IP (with kernel arguments ip=${net0/ip}:...):
- CentOS/RHEL distribution show "network unreachable"
This guide is intended to be a short writeup of what i learned about booting from iscsi using ipxe.
At the time of writing this some time has passed since i implemented these settings on my systems, so a few details might be incorrect or missing, always check documentation yourself.. i'll try to include as many reference links i can remember.
The goal of my setup was to implement remote booting of diskless hypervisor nodes running dracut or debian based distro's, with the specific requirement that no host/node specific settings like ip addresses, paths, iscsi initiators or targets, need to be configured on the diskless node.
This enables setting up a infrastructure where physical baremetal nodes can be added and removed from a cluster or any setup requiring no configuration other than setting the node bios to boot from nic via pxe/ipxe, and adding the MAC address of the booting nic to your DHCP server. The name of the iscsi initiator and target and all ip settings are configured only once in ipxe menu, after which they automatically follow through the whole boot process automatically thanks to ibft. A standard iscsi boot process requires 4 logins to the iscsi target hosting the root fs.
After that the adminitrator has full control of the node remotely, using ssh, wake-on-lan magic packets, ipxe MAC/hostname based boot menu defaults, and iscsi target filtering/LUN management, administrator can instruct the node to boot from any type of block device that can be exposed via iscsi, and set different boot defaults based on various conditions using any shell scripts, and or ipxe scripting, never needing to physically visit the node.
After hundreds of reboots i can say that these methods when working correctly are 100% predictable from the viewpoint of the remote admin, meaning that every single boot has been successful and no non-reproducable booting errors or failures have occurred yet, barring hardware failures.
IMPORTANT NOTE:
--------------
Using other than intel brand nic's can mess with nic interface ordering, especially in debian. If you have nic brands other than intel, and need to boot debian, you must read the important note at the end of the guide in addition to the debian section.
--------------
STEP 1: Set up robinsmidsrod's ipxe scripts at https://gist.github.com/robinsmidsrod/2234639. Ask in #ipxe @ freenode or this thread if you have issues. You need to define the variables and defaults, and have the /boot directory for adding node defaults. Don't worry, that's actually the hard part. Setting up debian and dracut is the easy part Image may be NSFW. Clik here to view.
Debian Jessie and Wheezy based distros:
Jessie:
1. Install standard debian distro, it doesn't matter how: bootstrap, install to iscsi target via debian installer, install to iscsi backed KVM volume, install to raw file, install to disk. As long as you cen export the resulting block device via iscsi you're good.
Easiest way is to start debian netinstaller in KVM/qemu VM with your working iscsi LUN target directly attached as kvm block storage.
(you can also run the installer with no local VM disk configured, only difference is you need to manually login to iscsi target after install, mount it, chroot into it to activate ibft booting)
2. After installation reboot into system and do as root:
^^^ IMPORTANT ^^^ A: this setting is only needed when you have more than one nic's in the system afaik, but i would set it anyway as system would otherwise become unbootable if more nics are added later. B: this setting is CRITICAL if you run more than one nic AND/OR ESPECIALLY if you run other than intel brand nic, in which case you must read important notice at the end of this guide. ^^^ IMPORTANT ^^^
3.
Code:
update-initramfs -u
Done, your system will now boot from iscsi with static or dhcp based ip settings, which are defined only once serverside in ipxe boot scripts.
Wheezy:
The procedure is exactly the same as above with the only exception that changes are needed in halt and reboot scripts to prevent them from hanging while waiting for iscsi initiator to disconnect before unmounting root, which ofcourse is impossible since root is on the remote target it wants to disconnect.
this is the part where te reader has to do his or her own googling, as i can't remember exactly the needed changes, and have no wheezy to test at time of writing. All i remember is one switch, probably -f (force) needed to be added to /etc/init.d/reboot , (command is 'reboot -d -f -i' in jessie), and in /etc/init.d/halt edit NETDOWN=yes
or no, just flip it, i can't remember what it was as default.
Dracut based:
1. Install as usual to any block device, as with debian. 2. after installation do: (replace yum with dnf for fedora i think)
(or grub-update or grub2-update or update-grub2, who cares anymore..)
Code:
dracut -v
Done.
notes:
Note 1: Assigning a host name to MAC address in DHCP server will result in robinsmidsrod's script to use hostname in iscsi initiator name (for example iqn.2007-09.jp.ne.peach.istgt:debian), which in conjuncture with ibft further improves and beautifies (!) the infrastructure, hostname and MAC are the only variables needed to be configured only once on the backend for each new node added to the network. Without DHCP assigned hostname initiator address will have MAC address (iqn.2007-09.jp.ne.peach.istgt:XX:XX:XX:XX:XX:XX).
Either way, set ipxe defaults for each node in ipxe menu /boot dir, maintain multiple default boot profiles for all nodes using multiple directories with appropriate default entries, or simply keep defaults static and edit the LUN in menu.ipxe:, or do both and add automation with CGI/Perl scripting and publish your scripts..
Code:
:iscsi-node-01
echo Booting iscsi node 01 for ${initiator-iqn}
set base-iscsi iscsi:${iscsi-server}::::${base-iqn}
set root-path ${base-iscsi}:iscsi-LUN-01
sanboot ${root-path} || goto failed
goto start
Note 2: if you want to add redundancy to you iscsi booting, you can add a loop in ipxe undionly.kpxe, or custom ROM that keeps querying the tftp and/or http server hosting the ipxe rom and menu.ipxe like this:
Incase you have a powerout, when the power comes back on the DHCP/tftp/http servers might not be ready, the loop will allow the nodes to wait instead of failing to boot with a timeout. The DHCP loop only works if you can control how the BIOS handles DHCP timeouts incase DHCP server is unreachable. Only drawback is it causes a few second delay in script even if all servers are up.
Currently, i chainload undionly.kpxe for my integrated nics which can't be flashed, and i have replaced DHCP with ifopen to reduce total DHCP queries for the whole boot process to 1 (the nic PXE rom dhcp query), after which that ip follows up the chain using ibft. If chainloading ipxe and using DHCP like the example above, and no static host ip entry was defined on DHCP server, this would result in 2 DHCP leases even before we get to ipxe menu.
Note 3: chainloading undionly.ipxe from tftp, and setting all clients in the infrastructure to boot via iscsi this way, then defining a default to escape ipxe menu and continue BIOS boot for the nodes that are not booted from iscsi normally, would enable you to control the bootprocess of even the clients booting from local HDD's remotely if needed.
Note 4: For even more redundancy host the iscsi LUN's on a RBD or DRBD mirrored pool, set up tftp/http/iscsi servers for High Availability.
-----------IMPORTANT NOTICE------------ For example adding a broadcom based nic to a node with one integrated intel nic, will probably flip the order of the nics at grub kernel boot, making the broadcom nic eth0 and the intel nic eth1. You need to be aware of this because in that case you need to change which MAC address is granted the DHCP configured hostname at boot, and also because in that case ipxe will see the intel MAC first, but after grub phase the linux kernel will switch the interfaces and thus the MAC and ip address of the client logging into the iscsi target _after_ grub phase will be different, meaning 2 entries for each node is needed in SAN firewall and/or iscsi target LUN access permissions.
During my testing this affected only debian when using other than all intel nics. luckily you can instruct initramfs which interface to use in netbooting, and thus combined with MAC settings on DHCP server, it's possible to mitigate this issue, which can be a real pain otherwise. Remember when setting DEVICE= in /etc/initramfs-tools/initramfs.conf , that it is the device used AFTER grub kernel stage, meaning if you boot via PXE/IPXE using the integrated intel nic, that nic would be eth1 in initramfs.conf, and the other brand additional nic would be eth0. Thus, setting this to eth1 in a machine with more than one nics of which some are non intel, combined with flipping the MAC address assigned for the hostname on the DHCP server, would result in a boot process where ony one DHCP lease, and one ip address is used, keeping the configuration persistent across infra. Ofcourse, the most bestest solution is to only use intel nics.
-----------^^^^^^^^^^^-----------
We are exprimenting with PXE boot in out Apple lab using iPXE (Mac minis Late 2012).
Described problem may not be directly related to iPXE since it might be more of Apple firmware issue combined with pxelinux.
I've seen few Mac related post on this forum so someone could now what is going on.
iPXE is booting via Mac firmware lagacy MBR boot with BIOS emulation.
Boot looks like this: Mac Firmware -> iPXE -> downloads and runs lpxelinux.
There are several boot scenarios.
1) [works correctly] Booting iPXE once via. Apple firmware boot menu when holding down option key from iPXE on USB flash drive or HDD partition.
2) [works correctly] Setting iPXE permanently from HDD partion with bless --device /dev/diskX --setBoot --legacy
3) [problem] Booting iPXE once from HDD partion using --nextonly flag (bless --device /dev/diskX --setBoot --legacy --nextonly)
iPXE starts, gets IP address, dowloads lpxelinux but ends with error:
Failed to load ldlinux.c32.
Boot Failed: ... see http://i.imgur.com/NIF3GC3.jpg
Do you think is just Apple bug or there might be explanation for this?
NVRAM states
Code:
NVRAM default (boot OS X from Macintosh HD)
efi-boot-device <array><dict><key>MemoryType</key><integer size="64">0xb</integer><key>StartingAddress</key><integer size="64">0xff990000</integer><key>IOEFIDevicePathType</key><string>HardwareMemoryMapped</string><key>EndingAddress</key><integer size="64">0xffb2ffff</integer></dict><dict><key>IOEFIDevicePathType</key><string>MediaFirmwareVolumeFilePath</string><key>Guid</key><string>2B0585EB-D8B8-49A9-8B8C-E21B01AEF2B7</string></dict><dict><key>IOEFIBootOption</key><string>HD</string></dict></array>%00
efi-boot-device-data %01%03%18%00%0b%00%00%00%00%00%99%ff%00%00%00%00%ff%ff%b2%ff%00%00%00%00%04%06%14%00%eb%85%05+%b8%d8%a9I%8b%8c%e2%1b%01%ae%f2%b7%7f%ff%04%00
BootCampHD %02%01%0c%00%d0A%03%0a%00%00%00%00%01%01%06%00%02%1f%03%12%0a%00%01%00%00%00%00%00%7f%ff%04%00
NVRAM after bless --device /dev/disk0s2 --setBoot --nextonly --legacy (boot through iPXE once via MBR legacy mode)
I'm quite new to iPXE and need some newbie advice...
I'm unable to get an Intel NUC to boot from the local hard drive neither with the exit (hard drive is second in the boot order) nor by using sanboot --no-describe --drive 0x80
I also tried --drive 0x81...5
It works though on a Dell Laptop.
No matter what I seem to do, I can't get a Wyse Thin client to load or boot windows installer through WIMBOOT. I have all the file loaded into the server and I can load the windows install on other computers, but when my Wyse client tries to load it, i get
Entering bootmgr.exe with parameters at 0x2a390 and nothing happens from there, Is there something else that could get getting the way of it loading.
Good morning friends!
I have a questions:
1) ipxe.efi may work with native menu? if yes - please show me example and source build optinos. i want graphic menu like in bios version.
2)ipxe.efi support iso booting? i cant load memdisk (and any .c32 app) and don't know how to boot iso.
3) where i can find ipxe uefi menu example (boot debian/centos installers / iso / and wimboot)
i try fetch=http://192.168.0.2/pxe/boot/live/uefi/pmagic/pmodules/PMAGIC_2016_01_06.SQFS
(not working)
and root=/dev/nfs boot=live netboot=nfs nfsroot=192.168.0.2:/mnt/1TB/pxe/boot/live/uefi/pmagic
with nfs on my server
(not working too)
I am with the FOG team and try to help getting to properly boot iPXE on the HP X2 210 tablet - UEFI enabled, secure boot off.
Loading bzImage (tried 4.3.3 or 4.4.0, EFI_STUB=yes) and initrd seams fine (DEBUG=image_cmd,image,efi_image). File sizes match and imgstat looks good as well. See picture here: https://forums.fogproject.org/uploads/fi...59_pro.jpg
But booting it just sits there (15 minutes...) and nothing happens.
Edit: Thanks to Michael I now added DEBUG=efi_wrap and will report back as soon as we have new information. Posting this here as this might help others as well.
I have been searching and searching to find a cause for this but I am not finding anything, so I figured I would seek out some assistance.
I have two machines, one running Windows Server 2008 that is running the iSCSI initiator, dhcp server, and tftp server, and stores the drive image for the diskless client. The next machine is diskless, but uses ipxe to connect to the server and connect to the iSCSI drive. The process works, however it takes 2 minutes plus or minus 2 seconds, from the time I get the starting sanboot message, until the windows logon screen comes up.
I put wireshark on the server and logged while I started the diskless machine. I see the traffic start when ipxe gets its IP address from the DCHP server and traffic continues until the Windows logo appears on the screen. At this point all the traffic just stops.
Looking at the wireshark log, traffic is flowing for 20 seconds, then stops. The next message is at 43 seconds from the server to the client NOP IN, followed by 4 retransmits. Then at 49, 50, 51, 122, 123, 124, 125 seconds there are ARP requests asking about the clients IP address, then at 125.5 there is an iSCSI login command, then the next 20 seconds are a flurry of message traffic.
I just wondered what I should look for as to a cause of this 105 second delay with no traffic?
I tried to install Windows Server 2003 directly for iSCSI target, but i got this error in install: Image may be NSFW. Clik here to view.
I used http://ipxe.org/appnote/xp_2003_direct_install tutorial.
I'm attempting to migrate from syslinux 6.03 to iPXE primarily for both http downloads of the kernel and initrd files as well as UEFI wimboot support (which I have working and is working great).
So far I've managed to get a boot script setup with menus that's working great to boot ubuntu, sysrescuecd, and several flavors of winpe.
I'm running into problems getting memtest to work properly and was wondering if somebody could help me with my boot script.
Under Syslinux I used an entry like below to load the memtest (mt86plus) that's included with the ubuntu live cd.
LABEL memtest
menu label Memory test
kernel mt86plus
I've tried using several combinations in my iPXE script to load the same mt86plus binary, however I keep getting the following error after iPXE successfully pulls down the mt86plus file from the pxe server.
I've also tried to use the memtest iso image from their website with memdisk and I can get memtest to load properly on bios systems, however as one would expect it will not work under UEFI systems since memdisk is not supported on UEFI systems!
I'd really appreciate any insight that you guys can give me!
Andy
hello to all forum members
i try to network boot android via serva everythink goes fine except this error
wget error
connecting to 10.1.1.1
wget cant connect to remote host network unrechable
please help me with this i really need to do this
iPXE> ifstat
net0: 1c:65:9d:7f:4eImage may be NSFW. Clik here to view.3 using ar9285 on PCI02:00.0 (closed)
[Link:down, TX:28 TXE:0 RX:10 RXE:4]
[Link status: Unknown (http://ipxe.org/1a086101)]
iPXE> iwlist
Networks on net0:
[Sig] SSID BSSID Ch Crypt/Auth
------------------------------------------------------------------
...
[ 55] XDXDX_XedXod 00:09:9*:**:93Image may be NSFW. Clik here to view.c 2 WPA PSK
...
iPXE> set net0/ssid XDXDX_XedXod
iPXE> set net0/key &12341234
iPXE> dhcp
Waiting for link-up on net0....
802.11 0xd67fc found network XDXDX_XedXod (00:09:9*:**:93Image may be NSFW. Clik here to view.c)
802.11 0xd67fc authnticating with method 0
802.11 0xd67fc associating
WPA-PSK 0xdc1c8: derived PMK from passphrase '&12341234'
0016bed7 : 52 cc 15 2a 31 1d 89 5d-71 51 a7 df c1 02 10 65 : R..*1..]qQ.....e
0016bee7 : c4 79 5f 76 19 5c 48 1f-bb 46 3b 95 48 e1 22 0e : .y_v.\H..F;.H".
802.11 0xd67fc security handshaking
802.11 0xd67fc security handshaking
WPA 0xdb9d8: received 1/4, looks OK
WPA 0xdb9d8 A1 1c:65:9d:7f:4eImage may be NSFW. Clik here to view.3, A2 1c:65:9d:7f:4eImage may be NSFW. Clik here to view.3
WPA 0xdb9d8 Nonce1, Nonce2:
0016beb4 : 71 e4 87 a3 9e 11 d7 89-1a 60 40 08 eb 08 d6 e0 : [omitted]
0016bec4 : 79 35 5a 70 c9 8b db c1-29 c7 9c e3 ee ea 25 c3 : [omitted]
0016bed4 : 2a 39 4d bc d4 6e c0 2b-3e 2a f2 f5 41 d9 87 c7 : [omitted]
0016bee4 : 71 6a 4e 2e c7 dc 38 eb-be 4c 5e 54 28 4e e4 75 : [omitted]
WPA 0xdb9d8 PTK :
[omitted]
WPA 0xdb9d8:sending 2/4
802.11 0xd67fc security handshaking
802.11 0xd67fc security handshaking
802.11 0xd67fc disassociated: reason 14
....
I would like to perform the following steps:
1. write ipxe image to usb (YES)
2. plug usb drive into machine then boot (YES)
3. download an image from http server (YES)
4. write the image to another part of the usb drive (???)
5. boot the new image, which is a linux from scratch system (???)
Is this possible with ipxe? Or perhaps even with some mods to ipxe source? Thanks in advance.
I am using iPXE for a while now and I am having some problems with a machine.
When I change the IDE controller from IDE to AHCI the WinPE images freeze at:
emulating drive 0x81
entering bootmgr.exe with parameters at 0x02a390
All the other machines boot correctly with no problems at all.
I use a pre-built binary from rom-o-matic.eu with my own script.
I've tried different output files and the .pxe one boots but the hard drive is not recognized when Acronis or Ghost loads up in WinPE.
When i reboot the machine and enter the BIOS the hard drive does not get recognized anymore. To fix it i have to disconnect/connect the drive again.
Also i've tried to use different versions of wimboot and other bootmgr.exe files that sadly dont work either.
I am guessing this is a hardware related problem but maybe you guys got an idea?
The machine is a POS (Senortech) with an Aptio BIOS and a Realtek B01 D00 NIC.
When i set the protocol on IDE everything works flawless.
thanks in advance!
Is it possible, within an iPXE script, to base a conditional statement (or variable value) on whether or not a download protocol is available in the iPXE instance that is running?
IE: Use HTTP if available, if not, use a TFTP url for network booting, something like:
dhcp
isset ${19:integer} || goto noHttp
set url http://...
goto gotProto
:noHttp
set url tftp://....
:gotProto
echo Booting using ${url}
boot ${url}
Since DHCP option 19 is the feature indicator option for ipxe.http, but this doesn't seem to work, perhaps because the option is only set when iPXE is sending the DHCP request, but in this example it's testing the server response (?)
I know I could first try the HTTP and then fall back on the TFTP statement if it fails, but the output of this is messy.
I'm having a really hard time getting iPXE to work with linux. I've been doing PXE for a long time and have had no problems getting that to work with linux. I love the features of iPXE though and I'm trying to convert my entire PXE environment to iPXE. Rather than burn ROMs etc, I've just set up my DHCP server to pass the undionly.kpxe loader, so technically i'm chaining from PXE to iPXE.
One of the things I put into my configuration was a hook to chain back over to my old PXE configuration so I can boot things in transition. My unbuntu live cd environment is working great there. Here's my stanza in PXE, whic owrks perfectly.
set ns 192.168.15.25
set np /pxe/linsetup/ubuntu/14043dx64
set args boot=casper netboot=nfs nfsroot=${ns}:${np} ipv6.disable=1 hostname=ubuntu64
initrd nfs://${ns}:${np}/casper/initrd.lz || goto doError
kernel nfs://${ns}:${np}/casper/vmlinuz.efi ${args} || goto doError
boot
This does not work. It loads the vmlinuz.efi and initrd.lz just fine; during debugging I inserted "imgstat" and "prompt key" before the boot so I could check that the parameters were correct. Boot starts and a few seconds later I get a kernel panic VFS: Unable to mount root fs on unknown block(0,0)
I have read literally dozens of articles all around the web on this, including the app note (http://ipxe.org/appnote/ubuntu_live) and it seems that I'm doing everything right.
I've fiddled with the boot params quite a bit and found:
Adding root=/dev/nfs doesn't help, and shouldn't be required in this version of ubuntu
adding ip=dhcp does seem to get a bit further - it at least seems to attempt to load the network before the kernel panic
adding "ro" doesn't help.
adding -- to the end of the arguments doesn't help.
Anyone got any working examples with 14.04.x and a current build of ipxe? I actually started this with a 14.04.1 CD and updated to 14.04.3 hoping that would fix it but no luck.
![[Image: w2003.png]](http://185.112.156.62/w2003.png)
