Skip to content

VM ingestion #3606

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
DaanHoogland merged 64 commits into from
Feb 3, 2020
Merged

VM ingestion #3606

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
f0aee9b
VM ingestion feature allows CloudStack to discover, on-board, import …
shwstppr Aug 2, 2019
f9e3936
fixes
shwstppr Oct 1, 2019
db2c8ca
fixes
shwstppr Oct 3, 2019
89f0e28
Update cloudstack-common.install
yadvr Oct 3, 2019
f080008
fixes
shwstppr Oct 4, 2019
2cb3f4f
fixed failing unit test
shwstppr Oct 4, 2019
3432b17
fix - cleaned test
shwstppr Oct 4, 2019
3739e4d
fix for vmfs storage pool
shwstppr Oct 7, 2019
8dedca7
fix storage pool check
shwstppr Oct 7, 2019
7e1f281
fix zone wide storage pool check, controllers details
shwstppr Oct 7, 2019
254d318
fix cpuSpeed for unconstrained offering
shwstppr Oct 8, 2019
bfb8505
fix warnings and refactoring
shwstppr Oct 8, 2019
52d4148
fixes
shwstppr Oct 8, 2019
b2780a3
fix VM states search, host maintenance
shwstppr Oct 9, 2019
96db573
fix nic, ipaddress assignment
shwstppr Oct 9, 2019
1c483ec
fix host maintenance while import
shwstppr Oct 9, 2019
dda4a17
fix Ip assignment, error message
shwstppr Oct 9, 2019
51b1ef9
fix disk controller check
shwstppr Oct 9, 2019
bbd6589
fix script error
shwstppr Oct 10, 2019
3454a7b
fix network vlan error
shwstppr Oct 10, 2019
fbbf975
fix auto assign network message
shwstppr Oct 10, 2019
3c0140c
fix network cluster, host
shwstppr Oct 10, 2019
44756d5
fix network IP address checks
shwstppr Oct 10, 2019
3991d75
templateid param for importUnmanagedInstance API made optional
shwstppr Oct 10, 2019
4824e53
fix error message multiple data disk controllers
shwstppr Oct 11, 2019
8f17219
fix vm import with dummy ISO
shwstppr Oct 11, 2019
b435957
fix hostname error on auto assign NIC
shwstppr Oct 11, 2019
370d88f
fix missing guest OS mapping
shwstppr Oct 11, 2019
ffa90b9
refatorings - error message
shwstppr Oct 11, 2019
e6ab488
fix - ignore IP for L2 networks
shwstppr Oct 11, 2019
39fd3b5
fix unmanaged nic vlan check
shwstppr Oct 14, 2019
384122c
fix volume migration failure
shwstppr Oct 14, 2019
2a45661
changes for creating dummy template from import service
shwstppr Oct 14, 2019
2d6af83
fix service offering, storage pool migration
shwstppr Oct 14, 2019
8638d05
migration null check
shwstppr Oct 14, 2019
76b581f
fix volume migration
shwstppr Oct 14, 2019
c18b818
fix isolated networks
shwstppr Oct 15, 2019
3029da7
fix volume migration based on vm running state
shwstppr Oct 15, 2019
c74ca2e
fix NPE for projectid param in import
shwstppr Oct 15, 2019
38318e6
fix - raise error when domainid is without accout param for import
shwstppr Oct 15, 2019
df1be1e
fix storagepool datastore casting
shwstppr Oct 15, 2019
e105ccf
fix moveVMToUser for removed template
shwstppr Oct 15, 2019
8381d2a
fix - add cluster details in error
shwstppr Oct 17, 2019
81d189c
fix guest OS retrieval
shwstppr Oct 18, 2019
405675b
fix vm guest os retrieval based on os id
shwstppr Oct 18, 2019
0fddb50
revert all os mapper changes
shwstppr Oct 18, 2019
3e44fdb
fix
shwstppr Oct 18, 2019
1662579
fix guest os selection order
shwstppr Oct 19, 2019
19feb2d
Merge branch 'master' into feature-vm-ingestion
shwstppr Oct 21, 2019
2257f04
fix stopped VM import, migration
shwstppr Oct 21, 2019
6ece417
added missing license
shwstppr Nov 7, 2019
a713211
fixed static member naming
shwstppr Nov 7, 2019
6e1abad
addressed loggin related review comments
shwstppr Nov 18, 2019
1eccd00
formatting refactoring
shwstppr Nov 18, 2019
9d8411d
refactorings
shwstppr Nov 18, 2019
770f241
refactored importNic
shwstppr Nov 18, 2019
1f94c67
refactoring: defensive check
shwstppr Nov 27, 2019
fe5853f
changed ipAddress key to ip4Address
shwstppr Dec 2, 2019
9fbafe6
vm import, pvlan support
shwstppr Jan 2, 2020
64edc77
Merge branch 'master' into feature-vm-ingestion
shwstppr Jan 2, 2020
e282680
Merge branch 'master' into feature-vm-ingestion
shwstppr Jan 17, 2020
2c52a1b
change for VirtualMachine state changes
shwstppr Jan 17, 2020
08b2603
removed diskofferingid param from import API
shwstppr Jan 17, 2020
0ec876f
added null check for network broadcast uri
shwstppr Jan 20, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions api/src/main/java/com/cloud/event/EventTypes.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,13 @@
import java.util.HashMap;
import java.util.Map;

import org.apache.cloudstack.acl.Role;
import org.apache.cloudstack.acl.RolePermission;
import org.apache.cloudstack.annotation.Annotation;
import org.apache.cloudstack.config.Configuration;
import org.apache.cloudstack.ha.HAConfig;
import org.apache.cloudstack.usage.Usage;

import com.cloud.dc.DataCenter;
import com.cloud.dc.Pod;
import com.cloud.dc.StorageNetworkIpRange;
Expand Down Expand Up @@ -69,12 +76,6 @@
import com.cloud.vm.Nic;
import com.cloud.vm.NicSecondaryIp;
import com.cloud.vm.VirtualMachine;
import org.apache.cloudstack.acl.Role;
import org.apache.cloudstack.acl.RolePermission;
import org.apache.cloudstack.annotation.Annotation;
import org.apache.cloudstack.config.Configuration;
import org.apache.cloudstack.ha.HAConfig;
import org.apache.cloudstack.usage.Usage;

public class EventTypes {

Expand All @@ -96,6 +97,7 @@ public class EventTypes {
public static final String EVENT_VM_MOVE = "VM.MOVE";
public static final String EVENT_VM_RESTORE = "VM.RESTORE";
public static final String EVENT_VM_EXPUNGE = "VM.EXPUNGE";
public static final String EVENT_VM_IMPORT = "VM.IMPORT";

// Domain Router
public static final String EVENT_ROUTER_CREATE = "ROUTER.CREATE";
Expand Down Expand Up @@ -594,6 +596,7 @@ public class EventTypes {
entityEventDetails.put(EVENT_VM_MOVE, VirtualMachine.class);
entityEventDetails.put(EVENT_VM_RESTORE, VirtualMachine.class);
entityEventDetails.put(EVENT_VM_EXPUNGE, VirtualMachine.class);
entityEventDetails.put(EVENT_VM_IMPORT, VirtualMachine.class);

entityEventDetails.put(EVENT_ROUTER_CREATE, VirtualRouter.class);
entityEventDetails.put(EVENT_ROUTER_DESTROY, VirtualRouter.class);
Expand Down
4 changes: 4 additions & 0 deletions api/src/main/java/com/cloud/vm/UserVmService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -513,4 +513,8 @@ UserVm upgradeVirtualMachine(ScaleVMCmd cmd) throws ResourceUnavailableException

void collectVmNetworkStatistics (UserVm userVm);

UserVm importVM(final DataCenter zone, final Host host, final VirtualMachineTemplate template, final String instanceName, final String displayName, final Account owner, final String userData, final Account caller, final Boolean isDisplayVm, final String keyboard,
Copy link
Contributor

@DaanHoogland DaanHoogland Nov 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a importVM(ImportVMCmd) method as well? or some other solution with parameter object pattern? This is not readable in declaration and won't be better in use.

Copy link
Contributor Author

@shwstppr shwstppr Nov 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is importUnmanagedInstance(ImportUnmanagedInstanceCmd) method in https://github.com/apache/cloudstack/pull/3606/files#diff-3eaaf9de1645091dff197645123e669cR1018
This method is added in UserVmService to keep all user VM commit in the same place.
Let me know if there is need for refactoring.

Copy link
Contributor

@DaanHoogland DaanHoogland Nov 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needed no, but desirable yes. This is a service call so it is open to third party extensions.

final long accountId, final long userId, final ServiceOffering serviceOffering, final String sshPublicKey,
final String hostName, final HypervisorType hypervisorType, final Map<String, String> customParameters, final VirtualMachine.PowerState powerState) throws InsufficientCapacityException;

}
8 changes: 8 additions & 0 deletions api/src/main/java/com/cloud/vm/VmDetailConstants.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,4 +54,12 @@ public interface VmDetailConstants {
String SSH_PUBLIC_KEY = "SSH.PublicKey";
String PASSWORD = "password";
String ENCRYPTED_PASSWORD = "Encrypted.Password";

// VM import with nic, disk and custom params for custom compute offering
String NIC = "nic";
String NETWORK = "network";
String IP4_ADDRESS = "ip4Address";
String IP6_ADDRESS = "ip6Address";
String DISK = "disk";
String DISK_OFFERING = "diskOffering";
}
19 changes: 19 additions & 0 deletions api/src/main/java/org/apache/cloudstack/api/ApiConstants.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ public class ApiConstants {
public static final String ACCOUNT_TYPE = "accounttype";
public static final String ACCOUNT_ID = "accountid";
public static final String ACTIVITY = "activity";
public static final String ADAPTER_TYPE = "adaptertype";
public static final String ADDRESS = "address";
public static final String ALGORITHM = "algorithm";
public static final String ALLOCATED_ONLY = "allocatedonly";
Expand All @@ -43,16 +44,23 @@ public class ApiConstants {
public static final String BYTES_WRITE_RATE_MAX = "byteswriteratemax";
public static final String BYTES_WRITE_RATE_MAX_LENGTH = "byteswriteratemaxlength";
public static final String BYPASS_VLAN_OVERLAP_CHECK = "bypassvlanoverlapcheck";
public static final String CAPACITY = "capacity";
public static final String CATEGORY = "category";
public static final String CAN_REVERT = "canrevert";
public static final String CA_CERTIFICATES = "cacertificates";
public static final String CERTIFICATE = "certificate";
public static final String CERTIFICATE_CHAIN = "certchain";
public static final String CERTIFICATE_FINGERPRINT = "fingerprint";
public static final String CERTIFICATE_ID = "certid";
public static final String CONTROLLER = "controller";
public static final String CONTROLLER_UNIT = "controllerunit";
public static final String COPY_IMAGE_TAGS = "copyimagetags";
public static final String CSR = "csr";
public static final String PRIVATE_KEY = "privatekey";
public static final String DATASTORE_HOST = "datastorehost";
public static final String DATASTORE_NAME = "datastorename";
public static final String DATASTORE_PATH = "datastorepath";
public static final String DATASTORE_TYPE = "datastoretype";
public static final String DOMAIN_SUFFIX = "domainsuffix";
public static final String DNS_SEARCH_ORDER = "dnssearchorder";
public static final String CHAIN_INFO = "chaininfo";
Expand All @@ -69,6 +77,7 @@ public class ApiConstants {
public static final String COMMAND = "command";
public static final String CMD_EVENT_TYPE = "cmdeventtype";
public static final String COMPONENT = "component";
public static final String CPU_CORE_PER_SOCKET = "cpucorepersocket";
public static final String CPU_NUMBER = "cpunumber";
public static final String CPU_SPEED = "cpuspeed";
public static final String CPU_LOAD_AVERAGE = "cpuloadaverage";
Expand All @@ -91,6 +100,7 @@ public class ApiConstants {
public static final String DETAILS = "details";
public static final String DEVICE_ID = "deviceid";
public static final String DIRECT_DOWNLOAD = "directdownload";
public static final String DISK = "disk";
public static final String DISK_OFFERING_ID = "diskofferingid";
public static final String NEW_DISK_OFFERING_ID = "newdiskofferingid";
public static final String DISK_KBS_READ = "diskkbsread";
Expand Down Expand Up @@ -170,6 +180,7 @@ public class ApiConstants {
public static final String PREVIOUS_ACL_RULE_ID = "previousaclruleid";
public static final String NEXT_ACL_RULE_ID = "nextaclruleid";
public static final String MOVE_ACL_CONSISTENCY_HASH = "aclconsistencyhash";
public static final String IMAGE_PATH = "imagepath";
public static final String INTERNAL_DNS1 = "internaldns1";
public static final String INTERNAL_DNS2 = "internaldns2";
public static final String INTERVAL_TYPE = "intervaltype";
Expand All @@ -181,6 +192,7 @@ public class ApiConstants {
public static final String IOPS_WRITE_RATE_MAX = "iopswriteratemax";
public static final String IOPS_WRITE_RATE_MAX_LENGTH = "iopswriteratemaxlength";
public static final String IP_ADDRESS = "ipaddress";
public static final String IP_ADDRESSES = "ipaddresses";
public static final String IP6_ADDRESS = "ip6address";
public static final String IP_ADDRESS_ID = "ipaddressid";
public static final String IS_ASYNC = "isasync";
Expand Down Expand Up @@ -224,13 +236,17 @@ public class ApiConstants {
public static final String NETWORK_DOMAIN = "networkdomain";
public static final String NETMASK = "netmask";
public static final String NEW_NAME = "newname";
public static final String NIC = "nic";
public static final String NIC_NETWORK_LIST = "nicnetworklist";
public static final String NIC_IP_ADDRESS_LIST = "nicipaddresslist";
public static final String NUM_RETRIES = "numretries";
public static final String OFFER_HA = "offerha";
public static final String IS_SYSTEM_OFFERING = "issystem";
public static final String IS_DEFAULT_USE = "defaultuse";
public static final String OLD_FORMAT = "oldformat";
public static final String OP = "op";
public static final String OS_CATEGORY_ID = "oscategoryid";
public static final String OS_ID = "osid";
public static final String OS_TYPE_ID = "ostypeid";
public static final String OS_DISPLAY_NAME = "osdisplayname";
public static final String OS_NAME_FOR_HYPERVISOR = "osnameforhypervisor";
Expand All @@ -255,6 +271,7 @@ public class ApiConstants {
public static final String PORTAL = "portal";
public static final String PORTABLE_IP_ADDRESS = "portableipaddress";
public static final String PORT_FORWARDING_SERVICE_ID = "portforwardingserviceid";
public static final String POSITION = "position";
public static final String POST_URL = "postURL";
public static final String POWER_STATE = "powerstate";
public static final String PRIVATE_INTERFACE = "privateinterface";
Expand Down Expand Up @@ -357,6 +374,7 @@ public class ApiConstants {
public static final String REMOVE_VLAN = "removevlan";
public static final String VLAN_ID = "vlanid";
public static final String ISOLATED_PVLAN = "isolatedpvlan";
public static final String ISOLATED_PVLAN_TYPE = "isolatedpvlantype";
public static final String ISOLATION_URI = "isolationuri";
public static final String VM_AVAILABLE = "vmavailable";
public static final String VM_LIMIT = "vmlimit";
Expand Down Expand Up @@ -528,6 +546,7 @@ public class ApiConstants {
public static final String CUSTOM_DISK_OFF_MAX_SIZE = "customdiskofferingmaxsize";
public static final String DEFAULT_ZONE_ID = "defaultzoneid";
public static final String LIVE_MIGRATE = "livemigrate";
public static final String MIGRATE_ALLOWED = "migrateallowed";
public static final String MIGRATE_TO = "migrateto";
public static final String GUID = "guid";
public static final String VSWITCH_TYPE_GUEST_TRAFFIC = "guestvswitchtype";
Expand Down
Loading