You signed in with another tab or window.
8000
Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Comments and Documentation not sufficient for implementing new cloud environments. #251
We want to add support for AzureStack, Microsoft's hybrid cloud. However the documentation in the Wiki on doing this is not clear. Also, all the comments are TBD for each cloud operation.
Hello... yes, this documentation is a work in progress, certainly! Question: do you intend to add this adapters through libcloud or with a native python client?
Alright, I have started populating the short description in https://github.com/ibmcb/cbtool/wiki/FAQ-D with a little bit more detail, regarding the different methods. I was busy with other projects the past week, but should be free to quickly answer any questions that you have this coming week.
I have started a blank template here. The only question I have is how do you know what will is required/optional in obj_attr_list? Also, how do you know what to set in it as well?
Also, is there any documentation on the config/templates? I am not 100% sure on what some of the properties mean.
What is required and what is optional on the obj_attr_list is cloud-dependent. This dictionary gets populated in https://github.com/ibmcb/cbtool/blob/master/lib/operations/base_operations.py#L925, with the values that it reads from the config/templates. Documentation on the config/templates, is I will freely admit, non-existent at the moment (it has been on my todo list for a while).
changed the title [-]Comments and Documentation not sufficient for extension[/-][+]Comments and Documentation not sufficient for implementing new cloud environments.[/+]on Oct 23, 2018
So I "think" I have everything filled in but it is complaining with this,
unknown error: Your configuration file is missing the following configuration:
[USER-DEFINED : CLOUDOPTION_TESTAZSCLOUD]
AZS_LOGIN = XXXXX
Do you know what this is and where it comes from? I saw similar configuration type files under configs/regression but I don't see any for the current Azure configuration.
I am trying to convert the sizes in the Azure text file to more modern values, but I don't know what size:1gb is, in fact I can't find any reference to it in LibCloud.
Will make the change. Also, there are a few functions that are required but not listed in the Appendix as required.
get_ssh_keys
get_security_groups
create_ssh_key
EDIT:
Also, when I create a VM how can I tell if I should use a private vs. public IP address? I don't think I can create a public IP address for every VM it might want to create. Or does CBTool on create a single VM for each benchmarch?
I must also point out that, for the current libcloud-based Azure adapter, the behavior seems to be, when the first instance is created, a new Public IP gets allocated, and subsequent instances are SSH-accessible through a different port using the same Public IP. (e.g., https://github.com/ibmcb/cbtool/blob/master/lib/clouds/libcloud_common.py#L710)
Thanks, I will take a look. Does CBTOOL know to use the different ports? How would I report that information to CBTOOL?
I have some other questions on things that are not clear.
On first load you instantiate a cloud object and test the VMC connection and create some SSH keys. However, you don't reuse this object so I cannot store the keys in the object which requires me to save them in KeyVault. Is this the expected flow? EC2 automatically creates their keys so they don't have to worry about this as they are always available.
I am not sure what vmccleanup should do. It is called on the initial vmcregister but if I do cleanup all resources the SSH keys are lost and therefore cannot connect to any VMs. If I don't delete resources here where should I delete all my resources? If I am supposed to delete resources here how do I get SSH keys from CBTOOL?
It is not clear what an image means for the functions get_images/check_images. In Azure/AzureStack tenants cannot upload custom images, they can only upload VHDs and then attach VMs to those. I am also not sure on how once I prepare a VM I can make CBTOOL work, unless I overload CBTOOL to look for VHDs in a storage account.
When I create a VM with a public IP address and try to prepare it using cbssh I get an error that the IP address is blank. Is there some value I need to be setting? I am following EC2 pretty close.
Is there a way to automate CBTOOL including VM preparation?
Regarding the other question, here are my answers:
Hmmm... If I understood your question properly, I believe I had a similar problem writing the Google Compute Engine adapter. My solution on that case was to explicitly add the keys to the metadata section of each instance created... example: https://github.com/ibmcb/cbtool/blob/master/lib/clouds/gce_cloud_ops.py#L1017
vmccleanup should remove only Instances and Volumes (whose names have the "cb-" + obj_attr_list["username"] + '-' + obj_attr_list["cloud_name"].lower() prefix) left from previous executions (i.e., CBTOOL processes were killed and no vmdetach all/aiattach all were issued). It should not delete SSH keys.
You mean, they cannot upload bootable custom images? That seems so restrictive! Anyway, if there is a list of images obtainable through an API call, then purpose of check_images is to basically ensure - during the initial cloud attachment - that every image specified on the cloud configuration file exists (and has a corresponding UUID from the cloud provider), while the goal of get_images is to ensure, before an instance creation is issued, that the image UUID specified on the CBTOOL's VM attribute imageid1 is present on the cloud.
If the attributes cloud_ip, run_cloud_ip and prov_cloud_ip are populated, then you should have no problem invoking cbssh.
Activity
maugustosilva commentedon Oct 4, 2018
Hello... yes, this documentation is a work in progress, certainly! Question: do you intend to add this adapters through libcloud or with a native python client?
deathly809 commentedon Oct 4, 2018
Native python. I am trying to guess on what some of the functions do. And also what should be set and what you should expect to be passed in.
maugustosilva commentedon Oct 13, 2018
Alright, I have started populating the short description in https://github.com/ibmcb/cbtool/wiki/FAQ-D with a little bit more detail, regarding the different methods. I was busy with other projects the past week, but should be free to quickly answer any questions that you have this coming week.
deathly809 commentedon Oct 15, 2018
Awesome, I just back to work from Surgery so I will be in and out for the week. I will take a look and let you know if I have any questions.
deathly809 commentedon Oct 15, 2018
I have started a blank template here. The only question I have is how do you know what will is required/optional in obj_attr_list? Also, how do you know what to set in it as well?
Also, is there any documentation on the config/templates? I am not 100% sure on what some of the properties mean.
maugustosilva commentedon Oct 18, 2018
What is required and what is optional on the obj_attr_list is cloud-dependent. This dictionary gets populated in https://github.com/ibmcb/cbtool/blob/master/lib/operations/base_operations.py#L925, with the values that it reads from the
config/templates. Documentation on theconfig/templates, is I will freely admit, non-existent at the moment (it has been on my todo list for a while).[-]Comments and Documentation not sufficient for extension[/-][+]Comments and Documentation not sufficient for implementing new cloud environments.[/+]deathly809 commentedon Oct 25, 2018
So I "think" I have everything filled in but it is complaining with this,
Do you know what this is and where it comes from? I saw similar configuration type files under configs/regression but I don't see any for the current Azure configuration.
maugustosilva commentedon Oct 25, 2018
Hmmm... just making sure that you have a sane configuration. So, taking SoftLayer as an example... https://github.ibm.com/marcios/cloudbench/blob/master/configs/templates/_slr.txt... every attribute that has a value of
need_to_be_configured_by_user(e.g. https://github.ibm.com/marcios/cloudbench/blob/master/configs/templates/_slr.txt#L7) needs to be present on your private configuration file while trying to attach a cloud...For reference, here is the example private configuration file for a SoftLayer cloud: https://github.ibm.com/marcios/cloudbench/blob/master/configs/cloud_definitions.txt#L94
Does it help?
deathly809 commentedon Oct 25, 2018
I can't access those links.
maugustosilva commentedon Oct 25, 2018
Argh, my apologies, I used the links from our internal repo! Here it goes again, with the correct links:
So, taking SoftLayer as an example... https://github.com/ibmcb/cbtool/blob/master/configs/templates/_slr.txt ... every attribute that has a value of need_to_be_configured_by_user (e.g. https://github.com/ibmcb/cbtool/blob/master/configs/templates/_slr.txt#L7) needs to be present on your private configuration file while trying to attach a cloud...
For reference, here is the example private configuration file for a SoftLayer cloud: https://github.com/ibmcb/cbtool/blob/master/configs/cloud_definitions.txt#L85
Does it help?
deathly809 commentedon Oct 25, 2018
I filled those in for AzureStack [link]. Where does CLOUDOPTION_TESTAZSCLOUD come from?
maugustosilva commentedon Oct 25, 2018
That comes from https://github.com/deathly809/cbtool/blob/8ef87595f3ce9631e8a4d736c3d18c5607392288/configs/cloud_definitions.txt#L24
Also, make sure that the
STARTUP_CLOUDin https://github.com/deathly809/cbtool/blob/8ef87595f3ce9631e8a4d736c3d18c5607392288/configs/cloud_definitions.txt#L20 isMYAZSdeathly809 commentedon Oct 29, 2018
I am trying to convert the sizes in the Azure text file to more modern values, but I don't know what size:1gb is, in fact I can't find any reference to it in LibCloud.
maugustosilva commentedon Oct 30, 2018
A mistake (from my part, when I copied from Digital Ocean).... please use "Large"....
deathly809 commentedon Oct 30, 2018
Will make the change. Also, there are a few functions that are required but not listed in the Appendix as required.
EDIT:
Also, when I create a VM how can I tell if I should use a private vs. public IP address? I don't think I can create a public IP address for every VM it might want to create. Or does CBTool on create a single VM for each benchmarch?
maugustosilva commentedon Nov 9, 2018
Hello again. Thanks will add all 3 functions to the list of mandatory functions.
Regarding the public vs private IPs, please let me know if this answers your question: https://github.com/ibmcb/cbtool/wiki/FAQ-S#sq6
I must also point out that, for the current libcloud-based Azure adapter, the behavior seems to be, when the first instance is created, a new Public IP gets allocated, and subsequent instances are SSH-accessible through a different port using the same Public IP. (e.g., https://github.com/ibmcb/cbtool/blob/master/lib/clouds/libcloud_common.py#L710)
deathly809 commentedon Nov 9, 2018
Thanks, I will take a look. Does CBTOOL know to use the different ports? How would I report that information to CBTOOL?
I have some other questions on things that are not clear.
maugustosilva commentedon Nov 12, 2018
CBTOOL will know use whatever port in is specified on
prov_cloud_portto ssh into the instances... an example here https://github.com/ibmcb/cbtool/blob/master/lib/clouds/libcloud_common.py#L712Regarding the other question, here are my answers:
Hmmm... If I understood your question properly, I believe I had a similar problem writing the Google Compute Engine adapter. My solution on that case was to explicitly add the keys to the metadata section of each instance created... example: https://github.com/ibmcb/cbtool/blob/master/lib/clouds/gce_cloud_ops.py#L1017
vmccleanupshould remove only Instances and Volumes (whose names have the"cb-" + obj_attr_list["username"] + '-' + obj_attr_list["cloud_name"].lower()prefix) left from previous executions (i.e., CBTOOL processes were killed and novmdetach all/aiattach allwere issued). It should not delete SSH keys.You mean, they cannot upload bootable custom images? That seems so restrictive! Anyway, if there is a list of images obtainable through an API call, then purpose of
check_imagesis to basically ensure - during the initial cloud attachment - that every image specified on the cloud configuration file exists (and has a corresponding UUID from the cloud provider), while the goal ofget_imagesis to ensure, before an instance creation is issued, that the image UUID specified on the CBTOOL's VM attributeimageid1is present on the cloud.If the attributes
cloud_ip,run_cloud_ipandprov_cloud_ipare populated, then you should have no problem invokingcbssh.Hmmm... there is an automated installer for both Orchestrator and Workload images: https://github.com/ibmcb/cbtool/wiki/HOWTO:-Preparing-a-VM-to-be-used-with-CBTOOL-on-a-real-cloud